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EXECUTIVE  SUMMARY 


Acquisition  programs  have  had  varying  degrees  of  success  in  developing  and 
producing  weapon  systems  on  time  and  within  budget.  Today,  it  is  more  important  than 
ever  that  programs  to  acquire  systems  be  carefully  designed  and  managed.  The  results  of 
past  programs  hold  lessons  for  the  future. 

We  found  tliat,  on  average,  major  weapon  system  programs  cost  roughly  half  again 
as  much  as  originally  planned,  and  systems  took  one-third  longer  to  develop  than  planned. 
There  was,  of  course,  great  variability  in  these  measures  depending  on  program 
characteristics. 

The  equipment  types  with  the  highest  cost  growtli  were  tactical  munitions  and 
vehicles,  and  ships  had  the  lowest.  Vehicles  and  air-launched  tactical  munitions  had  the 
highest  development  schedule  growth,  and  tactical  aircraft  had  the  lowest.  There  was  little 
indication  that  outcomes  are  getting  substantially  better  or  worse  over  time,  although  they 
have  improved  since  the  1960s. 

For  most  weapon  types,  systems  that  had  a  predecessor  system  had  lower  cost 
growth,  because  they  usually  are  less  risky  technically.  However,  for  air-launched  tactical 
munitions,  vehicles,  and  electronic  aircraft,  modification  programs  were  often  as  difficult 
as  completely  new  ones. 

Prototyping,  multi-year  procurement,  and  development  contract  incentives  appeared 
to  help  to  reduce  cost  growth.  Early  implementation  of  a  design-to-cost  strategy  may  be 
useful,  but  quantitative  results  are  incomplete.  The  results  for  dual-sourcing  in  production 
were  mixed.  Fixed-price  development  and  total  package  procurement  were  unsuccessful. 

OBJECTIVE  AND  APPROACH 

How  well  have  past  systems  adhered  to  their  cost  and  schedule  plans?  Has  the 
institutionalization  of  the  acquisition  process  helped  or  hurt?  Have  there  been  some  times  in 
history  when  things  turned  out  better,  or  are  problems  inevitable?  What  aie  the  causes  of 
cost  overruns  and  schedule  slippages?  What  management  strategies  have  been  most 
successful?  What  kinds  of  systems  are  most  likely  to  have  trouble,  and  why?  What  can  the 
Defense  Department  do  to  improve  program  outcomes? 
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To  help  answer  some  of  these  questions,  IDA  developed  measures  of  cost  and 
schedule  growth  for  1 1  types  of  equipment  developed  since  the  1960s.  We  analyzed 
differences  in  the  measures  by  equipment,  by  time  period,  by  phase,  and  by  whether  the 
program  was  entirely  new  or  a  modification  of  an  earlier  system  (development  type).  We 
also  examined  the  effectiveness  of  six  management  initiatives  in  improving  these  outcome 
measures.  The  initiatives  assessed  were: 

•  Prototyping, 

•  Contract  incentives, 

•  Multi-year  procurement, 

•  Design-to-cost, 

•  Dual-sourcing,  and 

•  Total  package  procurement  and  fixed-price  development. 

IDA  analyzed  1 16  major  programs  that  are  part  of  the  reporting  system  set  up  by  the 
DoD,  the  Selected  Acquisition  Reports  (SARs).  In  each  case,  we  compared  the  results  of 
the  program  as  of  the  final  1989  SAR  with  the  plan  for  the  program  approved  at  the 
Milestone  II  meeting,  the  time  at  which  the  program  is  given  permission  to  proceed  into 
engineering  and  manufacturing  development  (EMD).  [For  most  of  the  historical  period  we 
examined,  this  phase  was  called  full-scale  development  (FSD),] 

The  database  includes  a  mix  of  aircraft  (tactical,  electronic,  bomber,  transport,  and 
helicopters),  tactical  munitions  (both  air-launched  and  surface-launched),  electronics 
programs,  strategic  missiles,  satellites,  vehicles,  and  ships.  We  set  minimum  data 
standards  for  programs  in  order  to  have  development  and  production  outcome  measures 
calculated.  One  hundred  programs  met  the  development  standard,  and  82  programs  met  the 
production  standard.  We  also  identified  the  management  strategies  applied  to  the  programs, 
with  special  attention  to  the  vehicles  and  the  ships  that  were  new  to  the  database. 

PATTERNS  IN  ACQUISITION  PROGRAM  OUTCOMES 

On  average,  total  program  cost  growth  was  47  percent — 29  percent  in  development 
and  56  percent  in  production.  Development  schedules  (Milestone  II  to  initial  operational 
capability)  grew  by  an  average  of  about  one-third  over  the  plan.  There  is,  of  course,  great 
variability  in  ihe.se  measuies  depending  on  program  characteristics. 

As  shown  in  Figure  ES- 1 ,  the  equipment  types  with  the  highest  total  program  cost 
growth  were  tactical  munitions  (10.^  percent)  and  vehicles  (96  percent).  The  lowest  cost 
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Note:  Numbers  in  parentheses  are  numbers  of  programs. 

Figure  ES-1.  Total  Program  Cost  Growth  by  Equipment  Type 

Development  schedules  are  important — the  system  should  be  ready  when  planned 
in  case  it  is  needed  for  combat.  The  equipment  types  with  the  highest  development  schedule 
growth  were  vehicles  and  air-launched  tactical  munitions,  while  the  type  with  the  lowest 
was  tactical  aircraft. 

We  looked  at  programs  by  time  period  in  order  to  determine  whether  particular 
times  yielded  particularly  favorable  acquisition  outcomes.  We  grouped  programs  into  time 
periods  according  to  their  FSD  start  years,  because  acquisition  strategies  are  often 
determined  by  that  point.  Figure  ES-2  shows  key  outcomes  by  time  period. 

There  is  little  indication  that  acquisition  program  outcomes  are  getting  either 
substantially  better  or  substantially  worse.  Outcomes  were  worst  in  the  1960s,  and  they 
have  improved  since  then.  However,  the  improvement  has  not  been  continuous.  Cost 
growth  remains  a  persistent  problem,  despite  improvements  in  management  procedures. 

The  jury  is  still  out  on  the  1980s-era  programs.  The  ten  programs  for  which  we 
have  sufficient  data  to  measure  total  program  cost  growth  are  staying  within  their  planned 
costs.  This  is  a  hopeful  sign.  However,  cost  growth  tends  to  accumulate  with  experience. 
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and  these  programs  are  incomplete.  Moreover,  development  schedule  growth,  a  key 
indicator  of  future  problems,  was  higher  in  the  1980s  than  in  the  1970s. 


1960s  Early  19708  Late  1970s  1980s 

H  Development  Cost  Growth  ■  Production  Cost  Growth  Q  Development  Schedule  Growth 

Note:  Numbers  In  parentheses  are  numbers  ol  programs. 

*  Only  10  cases  available— see  discussion  in  Chapter  IV. 

Figur*  ES>2.  Summary  of  Outcomes  by  Time  Period 

We  would  expect  that  programs  that  succeed  very  similar  systems  (dubbed 
modification  programs,  or  mods)  would  have  lower  cost  growth  than  completely  new 
programs.  Here,  there  were  some  surprising  results.  As  we  expected,  modifications  had 
lower  cost  growth  than  new  systems  overall.  For  some  equipment  types,  however,  the 
technical  difficulty  of  modification  programs  is  underestimated  just  as  much  as  that  of  new 
programs,  resulting  in  cost  growth.  This  growth  was  particularly  seen  in  air-launched 
tactical  munitions  in  development  and  for  vehicles  and  elecuonic  aircraft  in  production. 

RESULTS  FROM  ASSESSMENT  OF  ACQUISITION  INITIATIVES 

Acquisition  managers  should  consider  making  more  use  of  prototyping.  Prototyped 
programs  had  lower  mean  total  program  cost  growth  than  non-prototyped  programs. 
Prototyping  before  FSD  significantly  reduced  development  cost  growth  in  the  tactical 
munitions  programs,  where  the  programs  with  the  greatest  technical  risk  were  prototyped. 
The  major  goal  of  prototyping  is  to  reduce  technical  risk;  however,  it  also  reduces  cost 
growth,  particularly  for  technically  challenging  programs.  Figure  ES-3  shows  program 
outcomes  for  the  two  major  equipment  groups  that  used  prototyping  the  most. 
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Note;  Numbers  In  parentheses  are  numbers  of  programs. 


Figure  ES-3.  Outcomes  for  Programs  With  and  Without  Prototyping 


Among  aircraft  programs,  those  with  less  technical  risk  were  prototyped,  and 
prototyping  was  less  successful.  If  development  cost  can  be  made  more  predictable,  the 
DoD’s  credibility  would  be  enhanced  in  developing  support  for  further  investments  in  a 
program.  This  study  did  not  specifically  address  a  more  recent  idea,  the  increased  use  of 
advanced  technology  demonstrators  without  production.  More  study  is  needed  of  this  new 
strategy.  However,  our  results  do  have  one  implication  for  the  “new  prototyping”  issue. 
The  research  we  have  undertaken  shows  the  practical  value  of  the  SAR  reporting  system  in 
providing  data  for  monitoring  and  evaluating  acquisition  programs.  While  the  SARs  have 
imperfections,  they  are  extremely  valuable.  The  use  of  advanced  technology  demonstrators 
should  not  diminish  the  importance  of  cost  and  schedule  estimating,  reporting,  and 
monitoring. 

Well-designed  contract  incentives  are  an  inexpensive  way  to  induce  contractors  to 
reduce  costs.  Figure  ES-4  shows  development  cost  growth  for  programs  with  and  without 
FSD  contract  incentives  and  production  cost  growth  for  programs  with  and  without 
production  contract  incentives.  Development  cost  growth  is  significantly  lower  in  programs 
with  incentive  contracts  in  FSD.  For  incentive  contracts  in  production,  the  results  are  not 
statistically  significant,  but  different  contract  types  are  typically  used  at  different  stages  of 
production.  These  results  were  intuitively  appealing.  Much  more  information  is  needed  on 
how  to  design  and  time  incentives  so  that  they  work  best. 
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Figure  ES'4.  Cost  Growth  and  Contract  Incentives 

Multi-year  procurement  (MYP)  appears  to  be  a  successful  initiative,  as  practiced  by 
the  DoD  using  congressional  guidelines.  We  assigned  programs  to  one  of  four  groups, 
mature  MYP  programs  (those  with  three  or  more  years  of  MYP  experience),  immature 
MYP  programs  (those  with  current  MYP  contracts,  but  with  less  than  three  years  of 
experience),  MYP  candidates  (those  considered  but  not  approved),  and  programs  that  were 
not  official  candidates  for  MYP.  Figure  ES-5  compares  outcomes  among  the  four  groups 
of  programs.  Production  outcome  measures  are  considerably  better  for  the  mature  MYP 
programs  than  for  the  non-MYP  programs.  However,  our  results  do  not  guarantee  success 
if  multi-year  procurement  is  increased.  MYP  implies  some  protection  from  budgetary 
pressures,  and  it  is  impossible  to  offer  such  protection  to  all  programs.  Non-MYP 
programs  were  stretched  out  more  often.  To  a  certain  extent,  the  progr'’.ms  that  were 
procured  using  MYP  were  allowed  this  privilege  because  they  were  already  successfully 
managed. 

Design- to-cost  (DTC)  needs  to  be  implemented  early  to  be  successful;  otherwise,  it 
does  not  make  sense.  Figure  ES-6  shows  that  the  initial  implementation  of  DTC  in  the  early 
1970s  was  not  successful.  In  the  late  1970s,  DTC  programs  had  slightly  lower  cost 
growth,  and  it  is  too  early  to  tell  about  the  1980s  programs.  Our  case  analyses  and 
breakdowns  of  results  show  that  the  successful  implementations  of  DTC  are  those  that 
apply  the  initiative  early  in  the  demonstration/validation  phase,  before  design  decisions 
are  set. 
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Figure  ES-5.  Key  Variables  for  Programs  With  and  Without  MYP 
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Figure  ES-6.  Average  Cost  Growth  for  Programs  With  and  Without  DTC 

The  record  for  dual-sourcing  is  mixed.  Figure  ES-7  shows  the  outcomes  from  the 
full  sample  and  from  tactical  munitions  and  ships  (where  virtually  all  the  dual-sourcing  in 
our  group  of  programs  occurred). 

Both  equipment  types  show  improved  cost  growth  with  dual-sourcing.  However,  it 
appears  that  at  least  some  of  the  positive  effect  of  dual-sourcing  results  from  the  fact  that 
such  programs  are  less  likely  to  be  stretched.  Dual-sourcing  can  be  of  value  in  individual 
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cases.  In  the  current  environment,  when  it  is  going  to  be  difncult  for  the  government  to 
fund  even  one  source,  competition  should  be  considered  carefully.  In  evaluating  individual 
cases  of  dual-sourcing,  it  is  important  to  consider  the  cost  of  bringing  on  the  second 
source. 
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Muniticns  Programs  Munitkns  Programs 

■  With  Dual-Sourdrrg  ■  Without  Dual-Sourcing 


Figure  ES-7.  Outcomes  for  Programs  With  and  Without  Dual-Sourcing 

The  era  of  fixed-price  development  (FPD),  the  early  1980s,  is  now  over.  Our 
results  indicate  that  it  should  not  be  revived  (see  Figure  ES-8).  In  high-risk  programs,  the 
government  often  was  forced  to  reopen  the  contract  when  the  contractor  was  unable  to 
fulfill  the  terms  of  the  original  contract.  Cost  growth  was  higher  in  fixed-price  development 
programs,  and  problems  in  development  often  spilled  over  into  production.  This  initiative 
should  not  be  used  in  high-risk  programs. 

During  the  1960s,  the  government  tried  to  shift  risk  to  contractors  by  requiring 
contractors  to  bid  on  development  and  production  and  sometimes  support  work  under  one 
contract.  Cost  growth  in  both  development  and  production  is  substantially  higher  in 
programs  that  used  the  total  package  procurement  (TPP)  concept  (see  Figure  ES-9).  While 
the  goal  of  the  TPP  concept  was  desirable,  the  quantum  leap  in  acquisition  practice  that 
implementation  of  the  concept  represented  was  a  factor  in  its  failure. 

DoD  can  use  the  information  in  this  study  to  target  the  types  of  programs  that 
showed  the  highest  cost  and  schedule  growth — the  tactical  munitions,  vehicles,  and 
electronics/avionics  programs — for  increased  management  attention.  One  can  also  see  the 
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benefits  of  using  common  subsystems  from  the  outcomes  for  ships,  which  had  the  lowest 
cost  growth. 
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Figure  ES*6.  Outcomes  for  Programs  With  and  Without  FPD 
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Figure  ES-9.  Outcomes  for  Programs  With  and  Without  TPP 

When  making  decisions  about  future  programs,  DoD  can  use  the  cost  and  schedule 
records  of  past  programs  of  the  same  equipment  type,  or  programs  in  the  same  time  period, 
or  programs  using  similar  acquisition  strategies. 
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I.  INTRODUCTION  TO  VOLUME  I 


A .  BACKGROUND  AND  PROBLEM 

The  Department  of  Defense  (DoD)  faces  challenges  in  the  years  ahead  when  making 
decisions  about  defense  acquisition.  The  Soviet  threat  is  much  smaller  than  it  once  was, 
and  a  major  land  war  in  Europe  is  unlikely.  With  the  breakup  of  the  Soviet  Union,  the  need 
for  the  United  States  to  have  the  latest  technology  in  strategic  systems  is  less  urgent.  As  a 
counterbalancing  force,  the  United  States  now  faces  weaker  but  more  volatile  and  diffuse 
threats  rather  than  u  single  major  threat. 

As  a  result  of  these  changes  in  the  threat,  the  Defense  Department  expects  to  leceive 
less  funding  in  real  terms  for  future  weapon  systems.  At  the  same  time,  the  desire  for 
leading-edge  technology  has  pushed  the  cost  of  systems  higher.  Thus,  it  is  more  important 
than  ever  that  programs  to  acquire  systems  be  carefully  designed  and  managed.  The  results 
of  past  programs  hold  lessons  for  the  future. 

How  well  have  past  systems  adhered  to  their  cost  and  schedule  plans?  Has  the 
institutionalization  of  the  acquisition  process  helped  or  hurt?  Have  there  been  some  times  in 
history  when  things  turned  out  better,  or  are  problems  inevitable?  What  are  the  causes  of 
cost  overruns  and  schedule  slippages?  What  management  strategies  have  been  most 
successful?  What  kinds  of  systems  tire  most  likely  to  have  trouble,  and  why?  What  can  the 
Defense  Department  do  to  improve  program  outcomes? 

The  answers  to  questions  such  as  these  would  help  DoD  make  wise  decisions  when 
contracting  for  future  acquisitions. 

B.  OBJECTIVES 

To  answer  these  questions,  we  developed  the  following  objectives: 

•  Present  patterns  of  cost  and  schedule  expectations  and  outcomes  for  a  large 
group  of  major  programs,  including  ships  and  vehicles; 

•  Choose  a  .set  of  specific  management  initiatives  applied  to  acquisition 
programs,  and  describe  their  impact  on  cost  and  schedule  expectations  and 
outcomes;  and 


Assess  the  effectiveness  of  management  initiatives  and  provide 
recommendations. 


C.  APPROACH 

This  study  builds  on  work  done  for  a  prior  IDA  study  on  effective  initiatives  in 
acquiring  major  systems  [  1].‘  In  this  study,  we  build  on  the  database  created  for  that  study 
by  providing  an  additional  two  years  of  data  and  adding  ships  and  vehicles  to  the  database 
for  the  first  time. 

The  approach  we  took  to  attain  our  objectives  included  the  following  steps: 

•  Select  a  set  of  ship  and  vehicle  programs  to  add  to  the  acquisition  database, 

•  Collect  data  on  cost  and  schedule  outcomes  for  ship  and  vehicle  programs,  and 
update  the  other  programs  to  match, 

•  Gather  information  on  the  management  initiatives  applied  to  the  programs, 

•  Analyze  trends  in  cost  and  schedule  outcomes,  and 

•  Assess  the  effectiveness  of  management  initiatives  in  minimizing  cost  and 
schedule  growth. 

In  defining  program  outcomes  for  the  study,  we  focused  on  cost  and  schedule 
outcomes  rather  than  on  technical  performance.  Clearly,  the  ultimate  test  of  a  system  is  its 
performance  in  the  field.  Operation  Desert  Storm  was  a  case  in  point.  While  there  were 
some  problems,  U.S.  technology  generally  worked  well.  The  goals  that  are  most  often  not 
achieved  are  cost  and  schedule,  and  we  focus  our  attention  there. 

D.  OUTLINE  OF  REPORT 

Section  11  gives  a  historical  overview  of  defense  acquisition  policy.  In  Section  III, 
the  data  and  outcome  measures  used  in  the  study  are  described.  Section  IV  discusses  trends 
in  acquisition  program  cost  and  schedule  outcomes.  Sections  V  through  X  deal  with  the 
individual  management  initiatives.  Finally,  Section  XI  presents  conclusions  and  offers 
recommendations. 


*  References,  listed  at  the  end  of  this  volume,  are  referred  to  by  number  in  brackets. 
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II.  BACKGROUND  AND  ISSUES 


A.  THE  DEFENSE  ACQUISITION  PROCESS 

Systems  acquisition  in  the  DoD  encompasses  the  development  and  procurement  of 
weapon  systems  for  national  defense.  Acquisition  may  involve,  as  it  did  early  in  the  post- 
World  War  II  period,  the  selection  of  a  production  model  from  several  prototypes  and 
purchase  of  off-the-shelf  articles.  Alternatively,  it  is  common  to  have  a  long  process  of 
development  before  production.  Development  includes  early  conceptual  and  validation 
efforts  as  well  as  later  engineering  and  manufacturing  development.  Today,  development 
can  take  over  10  years. 

Based  on  administration  policies,  the  Department  of  Defense  develops  its  method 
for  accomplishing  national  security  policy  objectives.  DoD  assesses  the  capabilities  of 
existing  forces  and  the  resources  available  for  defense  in  the  context  of  current  and 
prospective  threats.  From  this  information,  it  develops  operational  requirements  and 
translates  those  requirements  into  operating  forces.  The  output  of  this  process  is  a  set  of 
operational  requirements  for  expansion,  modernization,  and  support  of  military  forces. 
These  requirements  are  translated  into  systems  that  are  acquired  through  a  military  service 
or  defense  agency  p  ogram,  following  a  specific  acquisition  strategy. 

Program  offices  are  established  within  DoD  to  refine  program  requirements,  to 
develop  and  acquire  weapon  systems,  and  to  integrate  them  into  operational  forces.  Military 
service  or  agency  program  managers  preside  over  the  process  within  DoD.  They  are 
responsible  for  interaction  with  their  superiors  and  with  the  Defense  Acquisition  Board 
(DAB)  and  the  Planning,  Programming,  and  Budgeting  System  (PPBS).  The  programs 
are  monitored  by  the  Office  of  the  Secretary  of  Defense  (OSD)  and  military 
service  management. 

B.  ERAS  IN  DEFENSE  ACQUISITION 

In  this  study,  we  want  to  determine  whether  outcomes  are  getting  better  or  whether 
cost  and  schedule  growth  are  persistent  problems.  We  discuss  four  time  periods;  the 
1960s,  the  early  197()s,  the  late  1970s,  and  the  1980s.  These  represent  different  eras 
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in  defense  acquisition.  Nelson  and  Tyson  [2]  give  a  fuller  historical  perspective  of 
defense  acquisition. 

During  the  1960s,  Secretary  of  Defense  Robert  McNamara  introduced  mechanisms 
for  planning  defense  acquisitions.  He  initiated  the  PPBS  and  the  Five-Year  Defense 
Program.  Systems  analysis  was  a  method  of  using  paper  studies  and  simulations  to  choose 
among  alternative  technologies  using  the  framework  of  cost-effectiveness  analysis. 
Management  became  centralized  within  OSD  rather  than  diffused  among  the  military 
services.  Concurrency  in  development  and  production  was  often  used  in  an  effort  to  speed 
the  process.  The  presumption  was  that  properly  planned  programs  would  proceed 
smoothly.  After  1965,  the  total  package  procurement  (TPP)  concept  was  used  in  an  effort 
to  reduce  the  government’s  cost  risks.  TPP  required  contractors  to  bid  on  the  development, 
production,  and  support  work  under  one  contract.  It  was  designed  to  thwart  contractors 
from  estimating  low  costs  for  development  (when  there  was  competition)  and  then  “getting 
rich”  on  sole-source  productions.  The  Selected  Acquisition  Reporting  system  was 
introduced  in  1968  to  summarize  cost,  schedule,  and  performance  data  on  major  systems. 

During  the  early  197()s,  there  was  a  negative  reaction  to  the  total  package  concept. 
A  number  of  total  package  programs  had  large  cost  oveiTuns,  and  some  contractors  had  to 
be  bailed  out.  Total  package  procurement  was  discontinued  in  favor  of  cost-plus-incentive- 
fee  contracts.  Deputy  Secretary  of  Defense  David  Packard  urged  doing  away  with 
concurrency.  A  more  cautious,  phased  process  under  which  programs  had  to  pass 
"milestone”  tests  before  they  could  move  on  to  the  next  phase  was  established  through  the 
Defense  Systems  Acquisition  Review  Council  (DSARC).  Prototyping,  testing  using  actual 
hardware  rather  than  paper  studies,  became  more  prevalent.  More  responsibility  for 
day-to-day  management  of  programs  was  delegated  to  professional  program  managers.  The 
Cost  Analysis  Improvement  Group  was  set  up  in  1972  to  establish  uniform  criteria  for  DoD 
Cost  estimates  and  to  support  the  DSARC  by  reviewing  the  military  services’  cost 
estimates.  DoD  instituted  the  design-to-cost  initiative  to  encourage  knowledgeable  cost- 
performance  tradeoffs  in  acquisition. 

During  the  late  1970s.  congressional  management  of  the  acquisition  process  greatly 
increased.  In  addition,  there  was  growing  concern  about  resource  constraints.  Extremely 
high  inflation  meant  that  budgets  that  were  planned  to  cover  reasonable  price  increases 
ended  up  buying  considerably  less  than  expected.  Program  stretchouts  became  a  common 
practice.  Rather  than  canceling  programs  for  which  substantial  investments  had  been  made 
in  development.  Congress  made  moderate  cuts  and  stretched  out  production.  This  had  the 
effect  of  increasing  unit  costs. 


II-2 


During  the  1980s,  a  major  buildup  in  acquisition  began  and  many  new  starts  were 
designed  to  exploit  new  technologies  rapidly.  The  Acquisition  Improvement  Program  (the 
so-called  Carlucci  initiatives  after  Deputy  Secretary  of  Defense,  Frank  C.  Carlucci)  was  a 
31 -point  plan  to  improve  the  acquisition  process.  These  points  can  be  grouped  into  five 
areas;  improve  general  management  principles,  increase  program  stability,  improve 
forecasting  and  information,  improve  support  and  readiness,  and  reduce  bureaucracy. 
Congress  later  added  a  32nd  initiative  calling  for  increased  competition.  Dual-sourcing  of 
major  systems  and  subsystems  was  encouraged,  and  greater  emphasis  was  placed  on 
operational  test  and  evaluation.  Two  conflicting  concerns  gained  prominence  in  the  1980s, 
a  sense  that  fraud,  waste,  and  abuse  needed  to  be  attacked  with  additional  auditing  and 
regulation,  along  with  the  concern  that  the  regulatory  and  administrative  burden  on 
contractors  was  contributing  to  high  costs. 

Since  the  1980s,  the  breakup  of  the  Soviet  Union  has  resulted  in  more  changes  to 
acquisition.  The  early  stages  of  weapon  system  development  are  being  given  higher  priority 
and  being  managed  more  aggressively.  An  expanded  DoD  science  and  technology  program 
will  allow  the  best  technologies  to  be  validated  through  the  construction  of  advanced 
technology  demonstrators.  Still,  keeping  systems  in  this  pre-Milestone  I  holding  zone 
allows  them  to  escape  the  scrutiny  given  to  systems  designated  as  “major.” 

Three  major  cycles  in  the  DoD’s  organization  have  emerged  since  World  War  II: 

•  Centralization  and  decentralization  of  broad  DoD  decision 
making.  In  both  the  Truman  and  Eisenhower  administrations,  the  tendency 
was  toward  centralization  of  functions  and  away  from  the  decision  making 
power  of  the  services.  These  steps  were  cautious  at  first — in  the  Truman 
administration,  the  idea  of  having  an  umbrella  organization  over  the  services 
was  new.  During  the  Eisenhower  administration,  the  Secretary  of  Defense  got 
the  power  to  hire  and  fire  staff.  The  Kennedy-Johnson  administration  further 
consolidated  power  within  the  Secretary’s  office  by  introducing  PPBS  and 
systems  analysis.  Up  to  that  point,  overall  DoD  organization  was  following  a 
trend  toward  centralization.  During  the  Nixon  and  Ford  administrations,  some 
power  was  given  back  to  the  services.  There  was  a  move  to  push  decision 
power  further  down  the  chain.  The  Carter  administration  moved  back  toward 
centralization.  Then  the  Reagan  administration  moved  to  decentralize  some 
authority. 

•  Centralization  and  decentralization  of  acquisition  decision  making 
between  OSD  and  the  services  and  differentiation  of  the 
bureaucracy  within  OSD.  Until  the  Eisenhower  administration,  the 
services  basically  did  their  own  procurement.  Then  an  acquisition  bureaucracy 
at  the  OSD  level  began  to  evolve.  Acquisition  evolved  from  a  simple  buy/no 
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buy  decision  to  a  two-phase  process — development  and  production.  The 
creation  of  the  post  of  Director  of  Defense  Research  and  Engineering 
(DDR&E)  in  1959  resulted  in  further  differentiation  of  the  OSD  bureaucracy. 
During  the  Nixon  administration,  Packard  tried  to  bring  parties  together  with 
the  DSARC  process.  The  DDR&E  gradually  became  stronger  and  gained  status 
during  the  Carter  administration  by  being  upgraded  to  an  under  secretary 
(USDRE)  and  being  designated  as  the  Defense  Acquisition  Executive  (DAE). 
The  Reagan  administration  broke  up  this  package  by  strengthening  the 
Acquisition  and  Logistics  Office,  which  then  vied  with  USDRE  for  status  as 
the  chief  acquisition  office — a  dispute  that  was  eventually  settled  when  Deputy 
Secretary  Taft  assumed  the  role  of  DAE.  The  Packard  Commission’s  rec¬ 
ommendations  included  the  establishment  of  an  acquisition  “czar”  to  oversee 
the  entire  process,  an  attempt  to  put  everything  back  together  again.  The 
appointees  to  this  position  [the  Under  Secretary  of  Defense  (Acquisition)]  have 
had  to  work  hard  to  exercise  the  authority  granted  to  them. 

•  Upgrading  and  downgrading  of  systems  analysis  since  Secretary 
McNamara.  Republican  administrations  have  tended  to  downgrade  the  level 
of  the  systems  analysis  function  to  a  directorate,  while  Democratic 
administrations  have  tended  to  raise  it  to  the  level  of  assistant  secretary.  This 
office  was  an  assistant  secretaryship  under  McNamara,  was  downgraded  to  a 
director  under  Nixon,  upgraded  to  an  assistant  secretary  under  Carter, 
downgraded  to  a  director  under  Reagan,  then  upgraded  late  in  the  Reagan  and 
Bush  administrations.  This  office  tended  to  challenge  programs  submitted  by 
the  military  services,  so  upgrading  its  status  implied  less  authority  for  the 
militai7  sei'vices  and  downgrading  it  implied  more. 

While  the.se  cycles  in  acquisition  are  important,  the  overwhelming  trend  over  the 
4()-year  period  has  been  toward  standardization  and  institutionalization  of  the  acquisition 
process.  During  the  immediate  post-war  period,  there  was  little  standardization  of  the 
acquisition  prcKCss.  Because  of  the  perceived  threat,  many  programs  were  fast-tracked.  By 
the  end  of  the  195().s,  in  order  to  start  a  program,  advocates  had  to  justify  it  through  a 
development  concept  paper  (DCP)  in  a  standard  format.  During  the  1960s,  the  acquisition 
process  became  more  institutionalized.  The  PPBS  and  systems  analysis  also  involved  more 
hurdles  for  a  program  to  get  through.  The  Nixon  administration  brought  increa.sed 
codification  of  acquisition  regulations.  The  Packard  initiatives  included  more  emphasis  on 
prototyping  and  more  autonomy  for  the  .services  in  the  execution  process.  During  the  Carter 
administration,  the  key  development  was  more  detailed  resource  allocation  from  the 
Defen.se  Resources  Board.  During  Reagan’s  administration,  a  key  initiative  was  the 
campaign  against  fraud,  waste,  and  abu.se. 
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C .  RECOMMENDATIONS  FOR  REFORM 


Since  the  beginning  of  DoD,  many  studies  have  been  done  by  panels, 
commissions,  and  other  bodies  on  how  major  systems  ought  to  be  acquired  and  on  related 
issues  such  as  the  organization  of  the  DoD.  These  bodies  can  play  several  different  roles. 
First,  they  can  be  catalysts  for  change.  For  example,  the  1972  Commission  on  Government 
Procurement  resulted  in  the  development  of  Circular  A-l()9,  which  codified  acquisition 
policy  for  the  entire  federal  government.  Panels  and  commissions  can  also  ratify  change 
that  has  already  happened.  For  example,  the  1970  Fitzhugh  Commission  recommended  that 
there  be  no  more  total  package  procurement,  when,  in  fact,  the  government  had  already 
decided  against  it.  They  can  also  be  used  to  enhance  the  visibility  of  change  that  has  already 
occurred.  Finally,  they  can  be  an  important  safety  valve.  When  officials  want  to  appear  to 
be  responsive  to  a  problem,  they  appoint  a  commission  to  study  it.  As  part  of  our  research, 
we  reviewed  the  recommendations  of  as  many  of  these  commissions,  panels,  and  studies 
as  we  could  find.  We  also  used  a  summary  of  the  recommendations  of  25  of  the  most 
important  studies  The  eight  major  bodies  we  examined  were; 

•  The  First  Hoover  Commission,  1949  [4], 

•  The  Rockefeller  Commission,  1953  [5], 

•  The  Second  Hoover  Commission.  1955  [6], 

•  The  Blue  Ribbon  Defense  Panel  (the  Fitzhugh  Commission),  1970  [7], 

•  The  Commission  on  Government  Procurement,  1972  [8], 

•  The  Defense  Resource  Management  Study,  1979  [9], 

•  The  Grace  Commission  (President’s  Private  Sector  Survey  on  Cost  Control), 
1984  [10],  and 

•  The  Packard  Commission  (Blue  Ribbon  Commission  on  Defense 
Management),  1986  [11], 

The  principal  issues  studied  by  these  and  other  panels  were  organizational 
responsibility  tor  acquisition  within  DoD  and  acquisition  strategies  and  techniques.  A 
common  feature  of  commissions  and  panels  is  the  tendency  to  identify  a  particular  problem 
and  then  recommend  an  office  be  established  to  oversee  or  solve  it.  Thus,  organizational 
responsibility  for  acquisition  has  changed  over  the  years. 

Some  recommendations  come  up  repeatedly.  For  example; 

•  Multi-year  contracting  was  recommended  by  five  different  groups  (the 
Symington  report  of  1960  [12],  the  Office  of  Federal  Procurement  Policy 
(OFPP)  study  of  1982  [13],  the  Defense  Science  Board  (DSB)  study  of  1983 
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[14],  the  Grace  Commission  of  1984  [10],  and  the  Packard  Commission  of 
1986  [11]). 

•  Limiting  the  number  of  programs  to  those  that  can  be  funded  was 
recommended  by  four  panels  (the  Acquisition  Advisory  Group  in  1975  [15], 
the  Acquisition  Cycle  Task  Force  in  1978  [16],  the  Affordable  Acquisition 
Approach  study  in  1983  [17],  and  the  Grace  Commission  in  1984  [10]). 

•  Simplification  of  the  process  was  a  common  theme  in  many  recommendations 
(the  Grace  Commission  recommended  that  contract  clauses,  the  regulatory 
system,  specifications,  and  contracting  in  general  be  simplified  [10];  the  OFPP 
study  in  1982  recommended  that  simplified  procedures  for  commercial 
procurement  be  developed  and  that  procedures  for  small  purchases  be 
simplified  [13];  and  the  Acquisition  Cycle  Task  Force  in  1978  recommended 
shortening  the  front  end  of  the  acquisition  cycle  [16]). 

•  Four  separate  studies  recommended  that  a  positive  career  path  be  established 
for  program  managers  (the  Fitzhugh  Commission  in  1970  [7],  a  RAND  study 
of  the  1970s  experience  [18],  the  DSB  in  1983  [14],  and  the  Grace 
Commi.ssion  in  1984  [10]). 

On  the  other  hand,  some  recommendations  regarding  acquisition  strategy  have 
varied.  During  the  1970s  and  1980s,  a  flurry  of  recommendations  were  made  promoting 
increased  competition.  Other  recommendations  include  preplanned  product  improvement, 
better  cost  analysis  and  early  warning  systems  for  cost  growth  and  schedule  slippage, 
independent  subsystem  development,  and  improved  planning  tools.  In  addition,  many 
recommendations  have  been  made  relating  to  overstated  requirements,  unnecessary 
regulations,  improving  technology,  operational  test  and  evaluation,  unnecessary 
bureaucratic  layers,  program  strategy,  and  resource  planning. 

The  following  summarizes  recommendations  with  respect  to  the  initiatives: 

•  MYP  was  recommended  frequently,  as  previously  discussed. 

•  Competition  (dual-sourcing)  was  recommended  by  the  Grace  Commission. 
Increased  commercial-style  competition  was  recommended  by  the  Packard 
Commi.ssion.  The  Acquisition  Cycle  Task  Force  study  and  RAND  [18]  called 
for  considering  competition  in  all  phases  of  the  acquisition  cycles. 

•  Prototyping  was  one  of  the  Packard  initiatives  and  was  recommended  by  the 
Packard  Commission. 

•  Design-to-cost  was  a  Packard  initiative,  but  was  not  explicitly  addressed  by 
any  of  the  groups. 

•  Total  package  procurement  should  be  prohibited,  according  to  the  Fitzhugh 
Commission. 

•  Incentive  contracting  was  not  fully  addressed  by  any  of  the  groups. 
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D.  ASSESSING  OUTCOMES  AND  THE  EFFECTIVENESS  OF 
INITIATIVES 

In  at  least  one  sense,  one  can  conclude  that  the  acquisition  process  has  been 
successful:  the  territory  of  the  United  States  has  not  been  attacked  since  World  War  II,  and 
the  deterrent  effect  of  our  weapon  systems  is  certainly  at  least  partially  responsible  for  this. 

The  emphasis  on  quantitative  assessment  of  program  outcomes  began  in  the 
McNamara  era  with  the  introduction  of  the  SAR  and  has  continued  since.  The  acquisition 
process  has  typically  been  assessed  in  terms  of: 

•  The  achieved  functional  performance  of  the  system  relative  to  the  requirements, 

•  The  meeting  of  planned  development  and  production  schedules,  and 

•  The  cost  of  the  system  relative  to  planned  cost  [18  through  24]. 

Congress  tries  to  cut  program  costs,  but  is  subject  to  pressure  from  constituents 
who  work  in  defense-related  industries.  The  temptations  for  Congress  to  compromise  by 
trimming  a  little  out  of  each  program  rather  than  canceling  whole  programs  are  enormous. 
Such  actions  fuel  cost  growth.  To  improve  the  predictability  of  programs,  there  have  been 
many  attempts  to  improve  the  quality  and  independence  of  the  cost-estimation  process. 
Cost  estimators  still  must  contend  with  changes  in  requirements,  schedules,  and  technical 
make-up,  as  well  as  economic  and  quantity  changes.  Also,  it  is  not  easy  to  quantify  the 
impact  of  major  technological  advances  on  cost. 

Development  and  production  schedules  have  always  been  a  matter  of  concern.  The 
tendency  is  to  underestimate  the  schedule  initially  in  order  to  get  a  program  going.  Again, 
as  with  cost,  technological  advance  and  its  impact  on  schedule  are  not  easy  to  quantify. 

Achieving  planned  functional  performance  is  the  goal  typically  given  highest 
priority.  Systems  have  generally  tended  to  meet  their  performance  goals  [19].  For  this 
study,  we  concentrated  on  evaluating  cost  and  schedule  outcomes,  because  cost  and 
-schedule  appear  to  pose  the  most  problems  for  the  acquisition  system. 

We  selected  six  initiatives  for  analysis: 

•  Prototyping  has  been  practiced  in  several  aircraft  programs,  in  missile 
programs,  and  in  avionics  programs.  Prototyping  is  designed  to  reduce 
technical  risk  by  building  and  testing  detailed  pieces  of  hardware  early.  The 
analytic  issue  is  whether  prototyping  results  in  more  predictable  costs  and 
schedules. 

•  Contract  incentives  are  frequently  used  to  induce  the  contractor  to  reduce  costs 
or  to  engage  in  other  behavior  beneficial  to  the  government.  Incentive  fee 
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contracts  typically  involve  a  cost  target,  and  the  contractor  splits  savings  or 
additional  costs  with  the  government  based  on  actual  costs.  Award  fee 
contracts  are  more  complex;  typically,  a  list  of  criteria  for  the  program  manager 
or  a  review  board  is  used  to  determine  the  fee  awarded. 

•  Multi-year  procurement  involves  committing  tne  government  to  a  procurement 
and  funding  plan  for  several  years,  in  the  hope  that  contractors  will  be  able  to 
produce  at  lower  cost  with  a  stable  plan.  The  analytic  issue  is  whether  the 
government’s  comrniunent  and  reduced  flexibility  result  in  a  cheaper  system. 

•  Design-to-cost  was  widely  practiced  in  the  1970s.  Design-to-cost  involves 
setting  a  cost  goal  vei7  early  on,  similarly  to  the  way  a  performance  goal  is  set. 
Progress  toward  meeting  the  cost  goal  is  reported  periodically.  In  this  report, 
we  discuss  how  design-to-cost  worked  in  practice  and  whether  there  is  any 
evidence  to  suggest  that  it  reduced  cost  growth. 

•  Dual-sourcing  has  been  practiced  for  years  in  subsystems  and  is  becoming 
increasingly  popular  in  major  systems,  particularly  in  missile  programs  and  in 
ships.  We  define  dual-sourcing  as  two  or  more  sources  in  production,  not  the 
competition  of  companies  for  FSD  or  production  contracts,  which  is  fairly 
routine.  Dual-sourcing  of  major  systems  often  requires  a  considerable 
inve.«,tment  in  technology  transfer  and  qualification.  The  analytic  issue  is 
whether  this  cost  is  recouped  in  a  less  expensive  total  system  and  whether 
savings  can  be  sustained  over  the  long  term  as  companies  become  accustomed 
to  dual-sourcing. 

•  Fixed-price  development  evolved  in  the  Navy  in  the  early  1980s  as  a  way  of 
forcing  contractors  to  share  some  of  the  risk  in  development.  The  programs  we 
studied  are  mostly  in  the  early  stages  and  we  do  not  have  final  outcomes; 
however,  we  were  able  to  examine  how  fixed-price  development  is  working  in 
practice.  As  a  companion  piece  to  fixed-price  development,  we  consider  the 
historical  experience  of  total  package  procurement,  which  forced  the  contractor 
to  share  the  risk  of  both  development  and  procurement. 
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III.  ACQUISITION  PROGRAM  DATA 


A.  PROGRAMS  IN  THE  SAMPLE 

We  selected  a  sample  of  1 16  acquisition  programs  for  analysis.  The  programs, 
listed  in  Table  III-l,  represent  the  follovi^ing  categories  of  equipment: 

•  Tactical  aircraft, 

•  Electronic  aircraft, 

•  Other  aircraft, 

•  Helicopters, 

•  Electronics/avionics, 

•  Strategic  missiles, 

•  Air-launched  tactical  munitions, 

•  Surt'ace-launched  tactical  munitions, 

•  Vehicles  and  ground  combat  equipment, 

•  Ships,  and 

•  Satellites. 

The  sample  includes  acquisitions  managed  by  the  Army,  Navy,  and  Air  Force  and 
both  programs  that  are  considered  successful  and  those  that  encountered  problems  that 
were  resolved  with  varying  degrees  of  success  (including  cancellation).  In  order  to 
investigate  differences  in  acquisition  program  outcomes  between  new  and  modified 
systems,  the  sample  contained  Loth  types  of  programs. 

The  sample  is  spread  over  approximately  32  years  when  grouped  by  FSD  start. 
Nearly  all  programs  in  the  sample  are  either  still  in  production  and  in  service,  or  are 
previous  versions  of  weapon  systems  that  are  still  in  production  or  in  service.  For  the 
development  analysis,  we  excluded  programs  fewer  than  three  years  past  the  start  of  full- 
scale  development,  leaving  1(K)  programs.  For  the  production  analysis,  we  excluded 
programs  with  fewer  than  three  years  of  production  experience,  leaving  82  programs. 
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Several  programs  were  difficult  to  classify  by  equipment  type  because  they  could 
reasonably  be  put  into  more  than  one  class.  AEGIS,  for  example,  is  an  electronic  system, 
but  also  shares  important  characteristics  with  the  ships. 

In  this  volume  of  the  report,  we  address  the  programs  in  the  equipment  types  as 
shown  in  Table  III-l.  However,  in  Volume  II,  we  highlight  two  special  groups  of 
programs:  ships  and  related  Navy  programs,  and  vehicles  and  related  Army  ground  combat 
programs.  These  programs  were  selected  for  special  emphasis  in  this  study.  In  the  second 
volume,  the  ship  electronics  programs — AEGIS,  AN/BSY-1,  and  AN/BSY-2 — are 
analyzed  with  the  ships.  In  fact,  their  outcomes  are  more  similar  to  the  ships  than  they  are 
to  the  other  electronics  programs,  and  their  production  costs  are  included  with  the  ships. 
Also  in  the  second  volume,  several  Army  ground  combat  programs — the  tactical  munitions 
MLRS,  Roland,  M-198  howitzer,  and  the  intelligence  program  ASAS/ENSCE — are 
analyzed  with  the  vehicles. 

B.  DATA  SOURCES 

For  each  of  the  programs  included  in  the  sample,  schedule  dates,  cost,  production 
quantities,  and  narrative  information  were  obtained  from  Selected  Acquisition  Reports 
(SARs),  and  the  latest  available  editions  of  the  Defense  Marketing  Service  (DMS)  “Missiles 
Market  Intelligence  Reports”  [25],  Jane ’s  Weapon  Systems  J 987-88  [26],  Jane 's  Armour 
and  Artillery  [27],  Jane’s  Fighting  Ships  [28],  the  Interavia  summary  of  weapons  [29],  and 
interviews  with  program  management  and  contractor  personnel.  The  SARs  were  used  as 
the  primary  source  of  information  because  they  are  official  government  documents. 

Development  estimates  (DEs)  of  schedules,  costs,  and  quantities,  made  at  Milestone 
II  or  at  the  start  of  full-scale  development,  were  obtained  from  the  earliest  available  SAR 
for  each  program.  (Because  some  of  the  acquisition  programs  predate  the  1967  initiation  of 
SARs,  their  “original”  estimates  of  schedules,  costs,  and  quantities  shown  in  this  report 
may  not  have  been  the  true  original  estimates;  they  may  instead  be  subsequent  revisions.) 
Current  estimates  (CEs)  of  schedule.s,  costs,  and  quantities  were  obtained  from  the  last 
year-end  SARs  for  completed  programs  and  from  the  December  1989  SAR  for  ongoing 
programs. 

The  December  1989  SAR  (or  the  final  SAR  for  completed  programs)  was  the  basis 
for  our  comparison  of  current  e.stimates  with  development  estimates.  The  December  SAR  is 
designated  the  comprehensive  annual  SAR;  it  is  important  because  it  coincides  with  the 
President’s  budget  submission  to  the  Congress.  Thus,  the  services  and  OSD  take  care  to 
ensure  that  the  SAR  data  contained  in  the  December  SARs  match  budget  items  and  the 
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Future-Years  Defense  Program  (FYDP).  Table  III-2  is  an  example  of  a  SAR  Milestone 
schedule,  and  Table  III-3  is  an  example  of  a  SAR  program’s  acquisition  cost  estimate. 


Table  ill-2.  Example  of  SAR  Schedule  Mileetonee 


9.  (U)  Schedule;  a.  Milestones 

Development  Estimate/ 
Approved  FYocram 

Current  Estimate 

Prototype  Seeker  Firings 

N/A  Jan  77 

Jan  77 

AIM/RIM-7M  FSD  (DSARC  II) 

Apr  78/Apr  78 

Apr  78 

Commence  Joint  TECHEVAL 

Feb  80/N/A 

Jun  80 

OSD  Program  Review 

Apr  80/N/A 

Aug  80 

Commence  lOT&E 

Apr  80/N/A 

Jun  81 

Approval  for  Service  Use 

May  81/N/A 

Nov  82 

DSARC  III 

Jun  81/N/A 

— 

IOC  (1st  delivery  to  Fleet) 

Jul  81/Jan  83 

Jan  83 

DNSARC  III 

-TNov  82 

Nov  82 

Source:  Reference  [30], 


Table  111-3.  Example  of  SAR  Program  Acquisition  Costs 


11.  (U)  Acquisition  Cost  (USN/IISAF):  (Current  Estimate  in  Millions  of  Dollars) 


Development  Estimate  Current  Estimate 

(FY  75-m _ Chances _ (FY  7.S-8D) 

a.  (U)  Cost 


Development  (RDT&E) 

.S4.5 

-1.2 

.53.3 

Procurement 

859.2 

+587.4 

1446.3 

G,  C&A 

(681.7) 

(+489.3) 

(1,171.0) 

Propulsion 

(46.7) 

(+14.0) 

(60.7) 

Odier  Hardware 

(35.8) 

(-13.4) 

(22.4) 

Procurement 

(66.4) 

(+19.1) 

(85.5) 

Total  Flyaway 

(830.6) 

(+509.0) 

(1,339.6) 

Fleet  Support 

(19.9) 

(+47.5) 

(67.4) 

Initial  Spares 

(8.7) 

(+20.9) 

(29.6) 

Conshuction 

— 

_ 

_ 

Total  FY  78  Base  Year  S 

913.7 

+576.2 

1.489.9 

Escalation 

344.4 

+924.3 

1,268.7 

Development  (RDT&E) 

(2.8) 

(+5.1) 

(7.9) 

Procurement 

(341.6) 

(+919.2) 

(1,260.8) 

Constfuction 

— 

_ 

— 

Total  Then- Years 

1,258.1 

+  1.. 500.5 

2,7.58.6 

b.  Quantities 

Development  (RDT&E) 

44 

— 

44 

Procurement 

11,095 

+4,497 

15..592 

Total 

11,1.39 

+4,497 

15..592 

c.  Unit  Cost 

Procurement; 

FY  78  Base- Year 

S.077 

+.017 

.094 

Tlien-Year 

.108 

+.069 

.177 

Program; 

FY  78  Base- Year  S 

.082 

+.015 

.097 

Then- Year 

.113 

+.067 

.180 

Source:  Reference  [30]. 


III-4 


The  S  ARs  present  cost  estimates  in  escalated  dollars  as  well  as  in  constant  dollars. 
Our  analysis  used  the  constant-dollar  estimates  so  that  inflation  would  not  distort 
comparisons  among  programs  whose  DEs  were  established  at  different  times.  For 
programs  that  were  rebaselined,  development  e.stimates  of  development  and  production 
costs  were  obtained  by  escalating  development  and  production  costs  at  the  time  of  the 
development  estimates  to  the  new  base-year  dollars  using  January  7.  1990  DoD  deflators. 

The  SAR  is  a  highly  aggregated  source  of  cost  information.  We  would  have 
preferred  data  sources  with  more  detail,  and  we  did  review  sources  such  as  the  Contractor 
Cost  Data  Reporting  (CCDR)  system.  Given  the  number  of  programs  in  our  sample  and  the 
timeframe  in  which  the  effort  was  to  be  accomplished,  we  opted  to  use  the  SAR.  The  SAR 
is  a  definitive,  standardized  source  of  data  with  visibility  at  decision  making  levels.  It  has  a 
prescribed  format  common  to  all  the  services  and  allows  for  comparisons  of  cost,  schedule, 
and  quantity  changes  across  programs.  Development  concept  papers  (DCPs)  were 
reviewed  to  provide  additional  cost  and  schedule  information  in  the  programs. 

Representatives  of  selected  contractors  and  program  offices  provided  additional  cost 
and  schedule  data  and  answered  questions  that  surfaced  during  review  of  the  SARs.  These 
interviews  greatly  enhanced  our  understanding  of  individual  programs. 

Narrative  information  was  obtained  on  the  applicability  of  various  defense 
acquisition  policies  and  initiatives  to  each  of  the  programs  included  in  the  sample. 
Information  was  also  obtained  where  available  concerning  the  nature  and  extent  of  any 
major  problems  that  were  encountered,  how  the  problems  were  managed,  and  what 
appeared  to  be  the  causes. 

C.  OUTCOME  MEASURES 

From  the  inlormation  we  gathered,  a  database  was  developed  to  allow  examination 
ol  program  outcomes  and  to  permit  analysis  of  the  effectiveness  of  acquisition  initiatives  in 
acquiring  major  systems.  One  measure  of  the  outcome  of  a  program  is  cost  growth  during 
development,  production,  and  across  the  total  program.  Another  indicator  of  good  program 
performance  is  the  extent  to  which  the  system  can  be  developed  and  produced  according  to 
plan.  Therefore,  we  also  viewed  schedule  slippage  in  development  and  production  as  a 
measure  ot  program  outcome.  Finally,  trends  in  quantity  change  give  clues  to  such  issues 
as  reasonableness  ot  the  development  plan,  the  degree  of  production  stability,  and  the 
prevalence  of  program  stretch. 
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The  specific  outcome  measures  produced  were  as  follows: 

•  Development  cost  growth  (DCG), 

•  Production  cost  growth  (PCG), 

•  Total  program  cost  growth  (TPCG), 

•  Development  schedule  growth  (DSG), 

•  Production  schedule  growth  (PSG), 

•  Development  quantity  growth  (DQG), 

•  Production  quantity  growth  (PQG),  and 

•  Stretch. 

We  use  the  word  “growth”  to  refer  to  changes  in  cost,  quantity,  and  schedule 
because  in  most  cases  the  change  reflects  an  increase  in  dollars,  numbers,  or  time. 
However,  in  a  few  cases  (such  as  when  development  or  production  quantities  are  reduced), 
“growth”  is  negative.  “Stretch”  is  a  measure  of  the  extent  to  which  production  rate  (quantity 
over  time)  is  decreased  or  stretched. 

For  each  of  the  measures  of  program  outcomes,  we  compared  the  program 
development  estimate  to  the  final  (or  current)  estimate.  We  defined  a  program  as  the  first 
major  version  of  a  weapon  system.  A  few  major  programs  maintain  the  same  designator 
and  a  con.sistent  set  of  SARs  throughout  several  versions,  the  most  prominent  example  is 
the  F-15  aircraft  program,  which  had  Milestone  II  in  1970  and  is  currently  in  production 
for  the  “E”  version.  This  creates  a  problem  as  to  which  costs  to  consider.  Since  our 
purpose  was  to  evaluate  alternative  acquisition  strategies  for  major  programs,  we  defined 
both  the  cost  measures  and  the  strategy  measures  for  the  first  version  of  the  system,  e.g., 
the  F-15  A.  We  did  include  later  versions  in  the  production  schedule  and  quantity  data 
where  they  were  included  in  the  same  SAR.  The  F-14  program  has  three  sets  of  SARs,  the 
F-14,  F-14A/D  and  F-14D.  We  developed  measures  for  two  programs,  the  F-14A  and  F- 
14D. 

As  an  example  of  a  typical  program’s  profile,  we  use  the  Hellfire  missile  program. 
Table  III-4  shows  cost,  schedule,  and  quantity  information,  extracted  principally  from  the 
SAR,  for  the  Hellfire  program.  The  development  estimate  information  is  from  the  initial 
Hellfire  SAR  of  June  1976.  The  current  estimate  for  the  Hellfire  program  is  from  the  most 
recent  SAR  available  at  the  time  of  this  study,  the  December  1989  SAR.  The  derivation  of 
“current  estimate  for  development  estimate  quantity”  is  described  in  the  next  subsection. 
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TabI*  IIU4.  Hallfira  Program  Schadula  and  Coat  Summary 


Develotanent  Estimate 
(6/76) 

Current  Estimate 
(12/89) 

Current  Estimate 
for  Development 
Estimate  Quantity 

Milestone  li 

2/76 

2/76 

Development  Start  Date 

12/72 

12/72 

_ 

Development  End  Date  (IOC) 

5/83 

7/86  (+44%) 

— 

Development  Quantity 

241 

333  (+38%) 

— 

Development  Cost  (M  $) 

210.3 

230.2  (+9%) 

— 

Milestone  III 

7/80 

3/82 

_ 

Pi'oduction  Stan  Date 

7/80 

3/82 

_ 

Production  End  Date 

9/86 

9/93 

Production  Quantity 

24,600 

56,716  (+130%) 

24,600 

Unit  One  Cost  (K  $) 

— 

— 

1,160.0 

Slope  of  Cost-(iuantity  Curve 

— 

— 

82.1%, 

Production  Cost  (M  $) 

277.9 

806.6 

475.1 

PAUC  (K  $) 

11.30 

14.22  (+26%) 

19.31  (+71%.) 

Total  Program  Cost  (M  $) 

488.2 

1,093.2 

705.27  (+44%) 

Total  PAUC  (K  $) 

20.46 

19.28  (-1%) 

28.67  (+40%) 

Years  of  Actual  Data 

Development 

Completed 

Production 

7 

Notes:  All  costs  arc  in  1975  dollars.  Numbers  in  parentheses  represent  percentage  change. 


A  summary  of  the  Hellfire  program’s  outcome  is  provided  in  Table  III-5.  The 
database  for  all  the  programs  used  in  this  study  is  found  in  Appendix  A. 


Table  lll-S.  Outcomes  for  the  Hellfire  Program 


Outcome  Measure 

Percentage 

Growth 

Development  Cost  Growth 

9 

Prcxluction  Cost  GrowUi*’ 

71 

Total  Program  Cost  Growth*’ 

44 

Development  Schedule  Growth 

44 

Prtxluction  Schedule  Growth 

75 

Development  Quantity  Growth 

-5 

Production  Quantity  Growth^ 

130 

Suetch 

-26 

Based  upon  the  increase  from  the  development 
estimate  quantity  to  the  current  estimate  quantity. 
Based  upon  the  current  estimate  of  the  cost  of  the 
program  quantity  contained  in  the  development 
estimate. 


1 .  Evaluation  of  Cost  Growth 

In  order  to  understand  outcomes  by  program  phase,  we  separated  cost  growth  into 
development  and  production  cost.  Since  production  cost  is  much  higher  than  development 
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cost,  it  tends  to  drive  our  estimate  of  total  program  cost  growth.  However,  development 
cost  growth  is  also  of  interest,  since  it  is  here  that  the  technical  challenges  are  met. 

The  techniques  applied  in  our  analysis  for  weapon  system  cost  growth  are  similar  to 
those  used  in  past  investigations  of  program  cost  outcomes  (for  example,  see  [1  and  19]). 
The  following  process  was  used  to  produce  development  cost  growth  ratios: 

•  All  program  cost  estimates  were  collected  in  the  base-year  dollars  specified  for 
the  program.  For  the  Hellfirc  program  example,  this  is  fiscal  year  1975  dollars. 

•  Actual  development  costs  were  determined  for  the  period  from  program  startup 
through  initial  operational  capability  (IOC)  date,  including  development  costs 
incurred  beyond  IOC  that  are  still  associated  with  the  original  research, 
development,  testing  and  evaluation  (RDT&E)  effort.  Development  costs  for 
major  modifications  and  other  changes  beyond  the  scope  of  the  original 
development  effort  were  excluded. 

•  The  development  cost  growth  (DCG)  ratios  were  calculated  by  dividing  actual 
development  cost  by  the  development  cost  estimate  at  Milestone  II.  For  the 
Hellfire  program,  this  is:  $230.2  million  divided  by  $210.3  million,  for  a 
development  cost  growth  factor  of  1.09.  We  report  the  cost  growth  as  a 
percentage  (e.g.,  development  cost  growth  for  Hellfire  is  9  percent.) 

Before  constructing  production  cost  growth  ratios,  we  had  to  address  some 
additional  issues.  First,  the  best  infomiation  available  from  the  SAR  is  the  annual  funding 
summary  that  appears  in  recent  SARs.  These  data  represent  the  price  to  the  government, 
not  strictly  the  cost  of  the  system  for  the  contractor’s  point  of  view.  In  this  effort,  “cost 
growth”  ratios  refer  to  the  price  or  cost  to  the  government. 

Second,  many  programs  change  their  planned  quantity  as  the  program  progresses 
through  production.  Therefore,  some  adjustment  to  costs  is  necessary  to  take  quantity 
change  into  account.  The  SARs  provide  estimates  of  cost  change  due  to  quantity  change 
(and  schedule,  engineering,  inflation,  and  estimating  changes).  We  did  not  use  these 
estimates,  because  we  found  that  program  offices  interpreted  the  guidelines  for  developing 
these  estimates  in  widely  divergent  ways.  Instead,  we  developed  price-improvement  curves 
from  the  SAR  annual  data  for  completed  production  years.  Price-improvement  curves  show 
the  relationship  between  cost  and  the  cumulative  number  of  units.  In  weapon  systems, 
early  units  tend  to  cost  the  most,  while  later  units  cost  less,  as  learning  takes  place.  From 
these  curves,  we  calculated  the  cost  of  the  originally  planned  quantity,  the  development 
estimate  quantity  (DEQ). 

In  the  Hellfire  example,  procurement  quantities  changed  from  24,6(X)  to  56,7 1 6,  a 
131-percent  increase.  Production  costs  increased  from  $277.9  million  to  $806.6  million. 
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Much  cf  this  growth  in  production  cost  had  to  be  due  to  more  than  the  doubling  of 
quantities  procured  and  not  to  cost  growth.  Using  our  price-improvement  methodology,  we 
estimated  actual  Hellfire  production  cost  of  the  originally  planned  24,600  missiles  as 
$475.1  million,  and  calculated  a  production  cost  growth  of  71  percent. 

Several  programs  examined  do  not  have  annual  funding  detail  in  the  SARs  that 
allow  calculation  of  the  current  production  estimate  at  the  development  estimate  quantity. 
When  no  detailed  data  were  available,  the  slope  of  the  price-improvement  curve  was 
assumed  to  be  90  percent.  The  current  estimates  of  production  cost  and  quantity  from  the 
SAR  were  used  to  estimate  first-unit  cost  and  production  cost  at  the  development  estimate 
quantity. 

IDA  estimates  of  total  production  costs  were  then  determined  by  adding  the  current 
estimate  of  development  costs  to  the  current  estimate  of  production  cost  at  the  development 
estimate  quantity.  In  the  Hellfire  program,  total  program  cost  at  the  DEQ  is  estimated  to  be 
$705.27  million.  The  total  program  cost  growth  is  then  44  percent. 

2 .  Evaluation  of  Program  Schedules 

We  also  report  estimates  of  schedule  .slippage  in  development  and  production. 
Systems  need  to  be  developed  on  an  appropriate  schedule  so  that  they  are  ready  for  users 
when  planned.  Thus,  development  schedule  growth  is  an  important  indicator  of  program 
success.  Production  schedule  slippage  is  more  complicated,  because  it  is  intertwined  with 
quantity  changes.  Production  schedule  growth,  with  quantity  constant,  often  means  that  the 
program  is  being  stretched  becau.se  of  cost  growth  or  because  of  funding  shortages. 
Production  schedule  growth,  with  increased  quantity,  often  means  that  the  program  is  more 
successful  than  anticipated. 

Schedule  growth  during  development  of  a  new  weapon  system  is  normally 
measured  by  the  amount  of  slippage  experienced  in  a  program  between  a  fixed  base  date 
(e.g..  Milestone  II  date  or  FSD  contract  start,  whichever  is  earliest)  to  its  completion.  After 
the  necessary  data  were  collected,  the  development  schedule  growth  (DSG)  ratio  was 
computed  using  the  following  foirnula; 

lievelopinent  Schedule  Growth  Ratio  =  Actual  Time  (Months)  from  FSD  to  IOC 

E.stiinaled  Time  (Month.s)  from  FSD  to  ICX? 

The  development  schedule  growth  for  Hellfire  is  44  percent. 
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Production  schedule  growth  is  determined  using  the  same  technique.  Production 
span  is  defined  as  the  period  from  Milestone  III  or  first  production  contract  to  production 
end  date  or  the  last  fiscal  year  of  planned  funding.  Production  schedule  growth  (PSG) 
ratios  are  computed  using  the  following  formula: 

_  „  .  Actual  Time  (Months)  from  Production  Start  to  Production  End 

E’roducuon  Schedule  Growth  Rauo  =  ■  . .  . 

Estimated  Time  (Months)  from  Production  Start  to  Production  End 


The  Hellfire  program  has  exhibited  production  schedule  growth  of  75  percent. 


3.  Evaluation  of  Quantity  Changes 

Both  development  quantity  and  production  quantity  changes  were  documented 
using  the  same  technique  as  described  for  program  schedules,  except  that  quantity  is 
substituted  for  time.  Hellfire  experienced  -5  percent  development  quantity  growth  (D(}G)- 
Production  quantity  growth  (PCX')  was  130  percent. 


We  developed  another  index  that  measures  the  extent  to  which  production  is 
stretched.  In  buying  major  weapon  systems,  the  government  frequently  finds  that  it  does 
not  have  a  large  enough  budget  to  buy,  say,  the  100  systems  per  year  that  it  had  planned.  A 
common  way  of  dealing  with  this  problem  is  to  buy  the  whole  quantity,  but  at  the  rate  of 
only,  say,  50  per  year.  To  measure  this  phenomenon,  we  express  the  following  ratio: 


PS 


Stretch  = 


DE 


PQ, 


The  Hellfire  program  experienced  a  stretch  of  -26  percent.  Systems  were  acquired 
relatively  quickly. 
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IV.  PATTERNS  IN  ACQUISITION  PROGRAM  OUTCOMES 


A.  INTRODUCTION 

This  section  presents  results  from  the  database.  Aggregate  outcomes  are  presented, 
and  differences  by  time  period,  equipment  type,  program  phase,  and  development  program 
type  (new  vs.  modification)  are  identified  and  discussed.  Table  IV- 1  shows  selected  results 
from  the  IDA  database. 


Table  IV-1. 

Statistics  on 

Key  Variables 

Outcome  Mea.sure 

Number  of 
Procrams 

Mean 

Unweichted 

Mean  CE 
WeichLs 

MeanDE 

Weights 

Development  Cost  Growth  (DCG) 

100 

45% 

29%> 

13%, 

Production  Cost  Growth  (PCG) 

82 

58% 

56%> 

32% 

Total  Program  Cost  Growth  (TPCG) 

82 

50% 

47% 

29% 

Development  Schedule  Growth  (DSG) 

100 

.'14% 

— 

— 

Production  Schedule  Growth  (PSG) 

77 

57% 

— 

— 

Development  Quantity  Growth  (DQG) 

98 

10% 

— 

— 

Prtxluction  Quantity  Growth  (PQG) 

82 

34%, 

— 

— 

Saetch 

76 

67% 

— 

— 

On  average,  unweighted  program  cost  grew  50  percent  over  planned,  45  percent  in 
development  and  58  percent  in  production.  Development  schedules,  an  important  indicator 
of  on-time  performance,  grew  by  about  one-third.  In  full-scale  development,  programs  had 
to  build  only  10  percent  more  systems  than  planned.  Production  schedules  grew  by  57 
percent,  but  this  was  largely  explained  by  a  34  percent  increase  in  production  quantity.  We 
found  that  stretch  was  a  more  useful  measure  v>f  production  schedule  and  quantity  growth; 
therefore,  we  u.se  stretch  instead  of  PSG  and  PC}G  in  the  remainder  of  the  paper. 

Note  that  cost  growth  means  are  shown  unweighted  and  weighted  by  two  measures 
of  program  size,  the  current  estimate  and  the  development  estimate.  Weighting  makes  a 
considerable  difference  in  average  cost  growth  magnitudes.  Average  total  program  cost 
growth  is  47  percent  using  CE  weights  and  29  percent  using  DE  weights.  In  this  section, 
we  show  key  cost  outcomes  using  both  types  of  weights.  In  later  chapters,  we  identify 
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those  instances  in  which  weighting  makes  a  diffei'ence  in  the  direction  of  a  result.  A 
discussion  of  weighting  issues  is  contained  in  Appendix  B. 


B  .  OUTCOMES  BY  TIME  PERIOD 

As  previously  discussed,  the  purpose  of  analyzing  outcomes  by  time  period  is  to 
see  whether  broad  program  policy  in  specific  time  periods  influenced  acquisition  outcomes. 
The  time  periods  analyzed  are  the  1960s,  the  early  1970s,  the  late  1970s,  and  the  1980s. 

Each  of  these  periods  had  different  acquisition  policies  and  initiatives.  In  the  1960s, 
the  idea  of  program  management  using  a  structured  milestone  process  was  just  beginning. 
Initiatives  used  included  total  package  procurement  and  concurrency.  Management  was 
centralized  within  OSD.  In  the  early  1970s,  the  prevalent  initiatives,  with  the  influence  of 
Deputy  Secretary  of  Defense  David  Packard,  included  incentive  contracting,  prototyping, 
and  design- tO'Cost.  In  the  late  1970s,  design-to-cost  became  institutionalized,  and 
experiments  with  dual-sourcing  in  tactical  munitions  were  tried.  In  the  1980s,  initiatives 
included  fixed-price  development,  multi-year  procurement,  and  more  dual-sourcing. 

We  grouped  programs  into  time  periods  according  to  their  FSD  starts,  because  FSD 
is  a  major  milestone  and  acquisition  strategies  are  often  determined  by  that  point. 
Therefore,  it  seems  reasonable  to  conclude  that  policies  at  the  time  of  FSD  have  the  most 
influence  on  a  program.  However,  a  typical  program  continues  for  over  ten  years  past 
FSD,  so  it  may  be  influenced  by  the  policies  of  other  periods  as  well. 

We  compare  observed  results  in  terms  of  cost  and  schedule  with  estimates  at  the 
time  of  full-scale  development.  Table  IV-2  shows  cost,  schedule,  and  quantity  outcomes  by 
time  period. 


Table  IV-2.  Summary  of  Outcomes  by  Time  Period  (Percent) 


Outcome 

Measure 

1960s 

Early  1970s 

Late  1970s 

1980s 

DCG 

40  (25) 

31(17) 

23  (32) 

26  (26) 

PCG 

78  (24) 

45(17) 

63  (31) 

TPCG 

61  (24) 

42(17) 

52  (31) 

DSG 

50  (25) 

25(171 

28  (32) 

32  (26) 

DQG 

17  (25) 

26(17) 

0(31) 

5  (25) 

Suietch 

111  (19) 

54(16) 

62  (31) 

18(10) 

Notes:  Cost  growth  figures  are  weighted  by  the  current  estimate.  Numbers  in  parentheses 


are  numbers  of  programs. 


The  1960s,  when  SAR  cost  estimation  was  in  its  infancy,  was  a  period  of  high  cost 
growth.  Major  programs  such  as  the  C-SA  aircraft  and  the  Minuteman  missile  were  being 
developed.  In  addition,  methods  of  tracking  and  managing  programs  were  less  highly 
structured  than  today  [31].  The  cost  growth  in  the  1960s  was  higher  than  in  later  periods. 
Production  cost  growth  and  total  program  cost  growth  were  significantly  higher. 
Development  schedule  growth  was  also  higher  in  the  1960s  than  in  later  periods. 

Programs  with  FSD  start  in  the  early  1970s,  the  time  of  the  Packard  initiatives,  had 
good  overall  records.  Cost  growth  in  production  was  relatively  low;  however,  the  number 
of  programs  started  in  this  time  period  was  also  relatively  low. 

Programs  with  FSD  starts  in  the  late  1970s  did  not  fare  as  well  (in  terms  of  cost 
growth)  as  those  that  began  in  the  early  1970s.  Their  overall  cost  growth  was  52  percent, 
nearly  as  high  as  the  1960s  high  of  61  percent.  Programs  did  well  in  development  cost  (23 
percent  growth),  but  less  well  in  production.  While  they  generally  did  well  in  terms  of 
meeting  their  development  schedules,  programs  were  likely  to  be  stretched  in  production. 
This  contributed  to  the  unfavorable  cost  outcomes. 

The  Jury  is  still  out  on  programs  begun  in  the  1980s.  Mean  development  cost 
growth  was  26  percent,  as  high  as  the  late  1970s.  However,  total  program  cost  growth  is 
- 1  percent,  and  this  is  significantly  lower  than  in  the  past  (significance  level  =  .05). 

Since  cost  growth  tends  to  accumulate  over  time,  we  expect  that  this  figure  will 
climb  with  increased  experience.  The  tendency  is  for  contractors  to  underestimate  costs 
initially.  As  the  program  progresses,  however,  program  managers  are  forced  to  request 
additional  funds.  Thus,  cost  growth  increases  as  experience  increases.  We  see  some 
evidence  of  this  worsening  trend  by  comparing  the  results  here  with  those  from  an  earlier 
IDA  study  based  on  1987  data.  At  that  time,  average  development  cost  growth  was 
16  percent,  as  compared  to  26  percent  now.  Moreover,  total  program  cost  growth  was  -8 
percent,  versus  '  1  percent  now.  In  addition,  development  schedule  growth,  an  important 
indicator  of  future  cost  growth,  has  increased  from  21  percent  to  32  percent. 

Because  cost  growth  accumulates  gradually  as  experience  is  gained,  cost  estimates 
have  to  be  revised  to  reflect  experience.  If  the  end  of  the  production  run  is  more  than  five 
years  into  the  future,  then  cost  estimates  for  the  out- years  would  not  appear  in  the  FVDP 
and  might  not  be  revised  immediately. 

Other  caveats  about  the  198()s  programs  include: 

•  The  relative  need  to  “sell”  a  program  at  a  given  time  may  influence  the  initial 
development  estimate  of  both  cost  and  schedule.  When  budgets  are  fairly 
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generous  and  expected  to  increase,  as  in  the  early  1980s,  obtaining  funds  is 
relatively  easy,  so  there  is  no  incentive  to  underestimate.  However,  if  budgets 
are  tight,  there  may  be  an  incentive  to  underestimate  costs  in  order  to  get  the 
program  funded.  This  would  lead  to  higher  cost  growth. 

•  Although  the  1980s  programs  appear  to  be  doing  well,  they  arc  not  very  far 
along.  These  programs  should  be  reevaluated  once  more  experience  has  been 
gained. 

C .  OUTCOMES  BY  EQUIPMENT  TYPE 

The  purpose  of  analyzing  outcomes  by  equipment  type  is  to  see  whether  outcomes 
are  substantially  different  for  the  various  classes  cf  systems  examined.  Table  IV- 3  shows 
outcomes  by  equipment  type. 

Tactical  munitions  programs  experienced  the  highest  total  program  cost  growth 
(103  percent  for  surface-launched  and  102  percent  for  air-launched)  of  any  type  of  system 
examined.  Tactical  munitions  probably  have  a  higher  percentage  of  technological  content 
than  other  weapons  systems.  The  guidance  and  control  system  usually  pushes  the  state  of 
the  an  and  represents  two-thirds  to  five-sixths  of  the  cost  of  the  total  system.  Tactical 
munitions  systems  are  not  very  glamorous  and  therefore  may  not  receive  as  much  high- 
level  management  attention  as  needed. 

Vehicle  programs  experienced  the  next  highest  cost  growth,  96  percent.  The 
average,  however,  masks  a  great  deal  of  variability  in  program  outcomes.  Additional  detail 
on  vehicles  and  related  programs  is  contained  in  Volume  II. 

Experiences  with  other  equipment  types  generally  were  much  better.  Aircraft, 
satellites,  and  strategic  missiles  tended  to  have  lower  total  program  cost  growth  than  tactical 
munitions. 

The  lowest  total  program  cost  growth  for  a  substantial  group  was  displayed  by  the 
ship  programs,  15  percent.  (Electronics/avionicr  cost  growth,  while  lower,  was  based  on 
minimal  data.)  While  development  cost  growth  was  not  particularly  low,  36  percent,  ship 
programs  tend  to  include  much  of  what  other  programs  define  as  development  costs  in 
production,  and  production  costs  did  not  grow  by  much.  Moreover,  ship  programs  tended 
to  procure  substantially  more  quantity  (99  percent)  than  planned.  Additional  detail  on  the 
ship  programs  is  provided  in  Volume  II.  Satellites  also  experienced  relatively  low  total 
program  cost  growth,  20  percent.  Increased  production  quantity  and  multi-year 
procurement  may  have  played  a  role  in  this  result. 
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Vehicles  had  the  highest  development  cost  growth,  104  percent,  but  we  had  only 
three  data  points.  Electronics/avionics  programs,  which  exhibited  the  next  highest  cost 
growth  in  development.  78  percent,  were  examined  only  for  development  cost  growth 
because  we  could  not  disaggregate  production  costs  from  the  SARs.  However,  the 
rationale  that  applies  to  the  history  of  cost  growth  for  munitions  programs  very  likely 
applies  to  electronics  programs  as  well.  Strategic  missiles  (S  percent)  had  the  lowest 
development  cost  growth. 

Development  schedule  growth,  as  measured  by  Milestone  II  to  IOC  actual  versus 
planned,  was  highest  for  air-launched  tactical  munitions  and  vehicles  programs  (both  at 
66  percent).  Tactical  aircraft  exhibited  the  least  development  schedule  growth  (5  percent). 
Overall,  aircraft  had  significantly  lower  development  schedule  growth,  and  tactical 
munitions  significantly  higher  DSG,  than  the  overall  sample  of  programs. 

D.  OUTCOMES  BY  PROGRAM  PHASE 

We  examined  cost  growth  in  development  and  in  production  separately  (as  shown 
in  Tables  IV- 1  through  IV-3).  From  Table  IV- 1,  we  can  see  that  cost  growth  is  less  on 
average  in  development  than  in  production.  This  may  be  because  there  is  less  time  between 
the  estimate  and  the  actuals  in  development — by  the  time  production  is  completed,  by 
contrast,  the  development  estimate  may  be  15  years  old  or  more.  Moreover,  the 
development  budget  (typically  represented  by  the  RDT&E  budget  line  item)  is  limited,  and 
development  activities  that  cannot  be  finished  by  the  time  this  budget  is  spent  may  have  to 
be  financed  under  the  production  line  item,  thus  increasing  production  cost  growth. 

However,  for  several  equipment  types  (Table  IV-3),  development  cost  growth  is 
greater  than  production  cost  growth.  These  include  electronic  aircraft  (33  versus  31 
percent),  vehicles  (104  versus  93  percent),  ships  (36  versus  14  percent),  and  satellites  (32 
versus  18  percent).  (Again,  we  regard  the  low  production  cost  growth  in  electronics/ 
avionics  as  an  accounting  anomaly.)  The  difference  for  electronic  aircraft  is  small,  and,  for 
ships,  development  costs  are  a  small  proportion  of  the  total.  Therefore,  the  two  most 
interesting  cases  are  the  vehicles  and  the  satellites.  Vehicles  tend  to  suffer  from  problems  of 
definition,  and  this  would  tend  to  increase  cost  growth  in  development.  Satellites  are  often 
technically  sophisticated,  which  would  tend  to  increase  development  cost  risk,  and  multi¬ 
year  procurement  may  have  helped  to  hold  down  production  cost  growth. 


E.  OUTCOMES  BY  DEVELOPMENT  PROGRAM  TYPE 


Finally,  we  analyzed  program  outcomes  for  both  new  development  programs  and 
modification  programs.  The  purpose  of  this  analysis  was  to  see  whether  outcomes  are 
substantially  different  between  new  and  modification  programs.  Table  IV-4  shows 
outcomes  for  new  and  modification  programs.  Table  IV-S  shows  additional  detail  by 
equipment  type. 


TabI*  IV-4.  Summary  of  Outcomas  by  Program  Typa  (Parcant) 


Outcome  Measure 

New  Pmerams 

Modification 

ProRrams 

DCCi(CE) 

33  (70) 

20  (30) 

DCG(DE) 

14  (70) 

10  (30) 

PCG  (CE) 

71  (55) 

23  (27) 

PCG  (DE) 

44  (55) 

12  (27) 

TPCG  (CE) 

59  (55) 

20  (27) 

TPCG  (DE) 

37  (55) 

14  (27) 

DSG 

34  (70) 

36  (30) 

DOG 

2(68) 

27  (30) 

Saetch 

75  (51) 

50  (25) 

Notes:  Numbers  in  parentheses  are  numbers  of  programs.  Cost  growth 


figures  are  weighted  by  the  current  estimate. 


We  would  expect  that  modification  programs  would  have  lower  technical  risk  and 
thus  less  cost  and  schedule  growth.  This  is  generally  the  case  in  the  aggregate  for  some 
equipment  types;  however,  some  modification  programs  exhibit  as  much  or  more 
development  cost  growth  as  new  ones.  This  is  true  for  tactical  and  electronic  aircraft  and 
particularly  for  air-launched  tactical  munitions  (156  percent  DCG  for  modifications  versus 
31  percent  for  new).  For  these  types  of  programs,  apparently  technical  risk  is 
underestimated  for  the  modification  programs.  Other  programs  show  patterns  more  like 
what  we  would  expect,  with  cost  growth  lower  for  modifications. 

In  the  aggregate,  we  saw  little  difference  in  development  schedule  growth  between 
new  and  modification  programs.  When  we  disaggregated  by  equipment  type,  development 
schedule  growth  was  much  as  we  would  expect — modification  programs  were  more  likely 
to  meet  their  schedule  goals,  or  to  overrun  them  by  less,  than  new  programs.  The  major 
exception  is  vehicles,  where  the  one  modification  program,  the  M60A2,  overran  its 
schedule  by  217  percent.  Surprisingly,  modification  programs  had  higher  development 
quanitity  growth,  largely  due  to  the  air-launched  tactical  munitions. 
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Table  IV-5.  Summary  of  Outcomes  by  Equipment  and  Program  Type  (Percent) 

EquiiMnentTvpe  Program  Type  DCG  PCG  TPCG  DSG  DOG  Stietdi 
TacUcal  Aircraft  New  (7)  21(6)  28(7)  27(7)  6(6)  12(6)  43(6) 
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With  respect  to  production  and  total  program  costs,  modification  programs 
generally  exhibited  less  cost  growth  than  new  programs.  There  are,  however,  two 
interesting  exceptions.  Electronic  aircraft  modification  programs  had  higher  overall  cost 
growth  than  new  ones,  possibly  because  of  technical  risk.  Recall  that  electronics  and 
avionics  programs,  a  big  part  of  electronic  aircraft,  had  the  highest  development  cost 
growth.  In  addition,  the  effort  required  to  integrate  the  electronic  equipment  with  the  rest  of 
the  airciaft  is  often  underestimated.  The  second  exception  is  vehicles,  which  are  frequently 
subject  to  considerable  change  in  technical  requirements.  With  only  three  vehicles  in  the 
database,  however,  it  is  difficult  to  draw  a  conclusion. 

F.  CONCLUSIONS 

There  is  little  indication  that  acquisition  program  outcomes  are  getting  either 
substantially  better  or  substantially  worse.  Development  schedule  growth  and  cost  growth 
in  development,  production,  and  the  total  program  remain  persistent  problems,  even  though 
considerable  improvements  have  been  made  in  the  information  available  to  program 
managers.  The  early  1970s,  the  time  of  the  Packard  initiatives,  seemed  to  have  better 
program  outcomes  than  the  1960s,  which  had  poor  outcomes. 

Our  conclusions  about  programs  begun  in  the  1980s  are  preliminary.  While  many 
programs  were  begun  in  the  1980s,  there  is  still  relatively  little  production  experience  to 
judge  how  they  have  fared.  Development  cost  growth  so  far  is  slightly  better  than  past 
experience,  while  development  schedule  growth  is  typical  of  past  experience.  Production 
outcomes  look  substantially  better  than  in  the  past.  However,  we  are  reluctant  to  draw  any 
conclusions  about  the  production  pha.se  because  of  the  small  number  of  programs  in  our 
sample  and  because  those  programs  arc  mostly  in  the  early  stages  of  production. 

Program  outcomes  differ  depending  on  equipment  type.  Tactical  munitions  and 
vehicle  programs  experienced  the  highest  total  program  cost  growth.  This  was 
foreshadowed  by  their  cost  and  schedule  problems  in  development.  Ship  programs  had  the 
lowest  total  program  cost  growth. 

Vehicle  programs  and  electronics  programs  had  the  highest  development  cost 
growth  of  any  equipment  type.  (We  were  unable  to  track  substantial  production  experience 
of  electronics/avionics  systems  due  to  data  limitation.s — production  data  is  usually  included 
in  the  platform  SARs  and  cannot  be  disaggregated.)  We  have  seen  that  problems  in 
development  tend  to  be  followed  by  production  problems.  This,  coupled  with  the  fact  that 
many  future  programs  emphasize  electronics  heavily,  suggests  that  these  programs  should 
be  targeted  for  increased  management  attention. 
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As  expected,  modification  programs  exhibited  lower  total  program  cost  than  new 
programs.  It  is  easier  to  stay  on  plan  for  a  modification  program.  However,  there  are  two 
equipment  types  for  which  this  was  not  the  case — vehicles  and  electronic  aircraft. 
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V.  PROTOTYPING 


A.  BACKGROUND 

Prototyping  has  been  practiced  in  some  form  or  other  throughout  the  history  of 
acquisition.  There  has  always  been  a  need  to  test  new  types  of  equipment  before  any  large- 
scale  application.  Since  the  end  of  World  War  II  and  the  advent  of  the  systems  management 
approach  to  defense  weapons  acquisition,  prototyping  has  become  a  more  prominent  part 
of  the  process.  During  this  period,  it  has  experienced  cycles  of  popularity  and  disfavor. 

The  extent  of  prototyping  is  roughly  counter-cyclical  with  the  DoD  budget.  When 
the  budget  is  ample,  there  is  little  prototyping,  and  when  the  budget  is  tight,  there  is  more. 
For  example,  there  was  considerable  prototyping  in  the  periods  of  build-down  after  the 
Second  World  War  and  the  Korean  War.  During  the  early  1960s,  there  was  little 
prototyping,  as  the  Kennedy  administration  believed  that  systems  analyses  could  take  the 
place  of  prototypes.  Less  than  a  third  of  major  systems  were  prototyped.  In  the  early 
1970s,  Deputy  Secretary  of  Defense  David  A.  Packard  emphasized  the  importance  of 
prototyping  in  a  tly-before-buy  strategy.  Around  half  of  major  systems  were  prototyped 
during  this  period.  During  the  early  1980s,  when  the  Reagan  buildup  occurred,  once  again 
the  defense  budget  increased  relative  to  GNP,  and  there  was  less  prototyping.  The  Packard 
Commission  report  in  1986  again  called  for  more  prototyping.  Recently,  the  emphasis  has 
been  on  prototyping  using  advanced  technology  demonstrators  to  use  resources  more 
efficiently. 

Consideration  of  prototyping  is  especially  timely  now  for  a  number  of  reasons: 

1.  A  decreasing  real  defense  budget  increases  pressure  on  weapon  system 
developers  to  make  their  programs  more  predictable  and  financially  viable. 
Prototyping  can  provide  these  benefits. 

2.  Asa  consequence  of  lower  overall  budgets,  less  funding  is  available  for  major 
acquisition  programs.  Prototyping  two  systems  is  often  cheaper  than  buying 
one. 

3 .  Fewer  new  starts  are  expected  in  this  lower  defense  budget  climate.  The  few 
new  programs  that  are  funded  are  likely  to  carry  a  great  deal  of  technical  risk 
and  to  push  the  state  of  the  art.  Since  chances  to  win  a  bid  are  becoming 
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increasingly  rare,  there  is  a  great  deal  of  pressure  to  underestimate  cost  and 
schedule.  Prototyping  encourages  realism  in  technology,  cost,  and  schedule. 

4 .  The  ability  of  the  government  and  contractors  to  sustain  the  defense  technology 
base  is  in  question.  If  not  enough  work  is  forthcoming  from  the  DoD,  then 
manufacturers  will  leave  the  industry.  More  importantly,  new  ideas  will  not  be 
forthcoming  from  the  technology  base,  and  design  teams  will  wither  away. 
Ben  Rich,  head  of  Lockheed’s  Advanced  Development  Projects  said,  “Kelly 
Johnson  [his  predecessor]  developed  47  different  airplanes  in  his  50  years.  In 
my  40  years,  I  developed  27  different  airplanes.  My  young  engineer  today  is 
going  to  be  lucky  to  see  one  project — an  ATF.”  [32]  Prototyping  can  help  to 
keep  design  teams  together. 

5.  Threats  to  national  security  are  changing  as  a  result  of  the  changes  in  Eastern 
Europe,  and  they  are  much  more  difficult  to  predict. 

6.  Technical  sophistication  is  increasing.  More  sophisticated  equipment  carries 
even  higher  technical  risk  and  risks  of  cost  and  schedule  growth.  Integration  is 
becoming  more  complicated.  Software  costs  are  becoming  a  major  part  of 
system  costs,  and  software  projects  have  been  difficult  in  the  past.  Making  the 
transition  from  design  to  production  is  also  a  major  concern,  particularly  if 
early  research  and  development  on  manufacturing  technologies  are  not 
addressed  by  the  defense  industry. 

The  evidence  on  prototyping  from  the  literature  consists  mainly  of  case  studies  and 
qualitative  observations.  This  section  addresses  the  issue  of  prototyping  using  quantitative 
evidence  where  available  for  prototype  and  non-prototype  programs.  It  draws  from  prior 
IDA  work  [33  through  36]. 

As  defined  in  this  study,  prototyping  refers  to  the  construction  and  testing  of 
working  models  created  to  demonstrate  specific  design  or  operational  objectives  in 
advanced  development  (but  not  in  concept  exploration),  e.g.,  before  engineering  and 
manufacturing  development  (EMD)  at  Milestone  II.  Our  definition  does  not  include  EMD 
test  articles. 

The  primary  objective  of  any  prototype  program  is  to  obtain  information  to  reduce 
the  uncertainty  and  risk  concerning  the  design  concept,  cost,  or  usefulness  of  a  particular 
model  before  committing  to  the  greater  expense  of  EMD.  Historically,  a  decision  to  go  into 
EMD  virtually  ensures  that  the  system  will  go  on  to  production. 

If  an  acquisition  program  is  to  be  successful,  potential  design  problems  need  to  be 
identified  and  resolved  as  early  as  possible.  Such  problems  can  affect  the  performance  and 
technical  characteristics  of  the  weapon,  its  development  schedule,  and  its  development, 
production,  and  suppon  costs. 
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The  primary  purpose  of  prototyping  is  to  reduce  technical  risk.  Prototypes  can  be 
used  to  answer  three  technical  questions.  The  three  questions,  which  are  not  mutually 
exclusive,  are: 

•  Is  the  concept  feasible? — Proof  of  Concept 

•  Does  the  design  work  the  way  it  is  supposed  to  work? — Proof  of  Design 

•  Does  the  system  provide  a  militarily  useful  capability? — Proof  of  Mission 
Suitability 

B.  BENEFITS  AND  WEAKNESSES 

A  key  benefit  of  prototyping  is  early  information — both  qualitative  and  quantitative. 
Before  committing  to  EMD,  the  government  determines  whether  or  not  a  design  can  meet 
the  objectives  specified  for  it  through  prototyping.  For  example,  can  a  vertical  takeoff  and 
landing  (VTOL)  aircraft  hover  in  controlled  flight?  Does  the  guidance  system  work?  Either 
a  positive  or  a  negative  answer  can  be  worthwhile.  If  the  system  does  not  work,  the 
designers  can  avoid  committing  to  a  fruitless  path  during  EMD,  when  much  greater  levels 
of  resources  are  committed  and  expended. 

Competitive  prototyping  can  also  be  very  useful  in  source  selection.  The 
government  may  make  a  better  choice  of  contractor  by  evaluating  testable  hardware  rather 
than  just  paper  studies.  If  there  is  a  teaming  arrangement,  prototyping  allows  the 
government  to  see  whether  or  not  contractor  teams  mesh  well  and  allows  anticipation  of 
any  integration  problems. 

Quantitative  information  from  prototyping  includes  performance,  schedule,  and  cost 
dimensions.  Required  performance  characteristics  can  be  validated  through  the  testing  of  a 
prototype,  or  the  requirements  can  be  changed  to  fit  what  can  reasonably  be  achieved. 
Acquisition  managers  can  also  make  more  credible  estimates  of  how  long  a  program  will 
take  and  how  much  it  is  likely  to  cost. 

All  of  the  information  gained  from  prototyping  may  result  in  lower  costs  for 
development  and  production  and  shorter  time  in  EMD  than  for  non-prototyped  systems. 

In  general,  weaknesses  of  prototyping  include  the  additional  resources  and  time 
needed  to  accomplish  a  prototype  prior  to  EMD  start.  The  cost  of  the  prototyping  phase  is 
an  upfront  cost,  and  there  is  often  considerable  resistance  to  committing  the  funds.  The  YF- 
16  prototype  cost  on  the  order  of  $1(K)  million  in  a  $30  billion  program;  the  A- 10,  about 
$100  million  in  a  $5  billion  program,  and  the  AV-8B,  $150  million  for  a  $10  billion 
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program.  The  cost  ranges  less  than  2  percent  of  the  total  program.  This  seems  to  be  the 
range  for  other  types  of  equipment  as  well. 

It  is  not  clear  that  prototyping  increases  program  costs  above  what  they  would  have 
been  without  a  prototype.  Without  prototyping,  costs  might  grow  even  more  as  problems 
crop  up  later  in  the  program,  when  spending  rates  are  high.  We  examine  some  evidence 
here. 

Prototyping  may  result  in  some  increase  in  time  to  achieve  initial  operating 
capability  (IOC).  Prototyping  involves  building  and  testing  hardware  at  an  early  stage. 
Some  program  officials  complain  that  prototyping  slows  down  the  momentum  of  the 
program. 

Prototyping  might  be  overkill  when  the  information  to  be  gained  is  not  all  that 
important  to  the  accomplishment  of  the  program.  This  can  be  a  difficult  decision.  One 
example  here  is  the  aircraft  nuclear  propulsion  program  in  the  1950s.  The  uncertainty  had 
to  do  with  the  aircraft  nuclear  reactor  performance,  weight,  and  cost.  However,  a  great  deal 
of  time  and  money  went  into  building  the  large  turbine  engine  that  was  going  to  be  driven 
by  the  nuclear  reactor,  even  though  the  engine  technology  was  well-understood.  It  is 
estimated  that  an  additional  $5(X)  million  was  spent  on  the  aircraft  nuclear  propulsion 
program,  because  the  X-21 1  turbine  engine  was  built  and  tested  despite  the  substantial 
technical  and  cost  risks  for  the  reactor.  The  program  was  eventually  canceled  because  of  the 
technical  problems  and  costs  associated  with  the  nuclear  reactor.  This  example  highlights 
the  need  for  sound  judgment  to  select  appropriate  prototypes,  at  the  subsystem  level  as  well 
as  at  the  system  level. 

C.  TYPICAL  APPLICATION 

Prototyping  may  be  accomplished  at  the  system  or  subsystem  level.  It  may  be  used 
during  concept  exploration  to  achieve  a  proof  of  concept,  or  during  advanced  development 
to  achieve  a  proof  of  performance,  cost,  or  operational  suitability.  An  example  at  the 
system  level  of  proof  of  concept  would  be  a  new  veriical/short  takeoff  and  landing 
(V/STOL)  aircraft  design  approach  such  as  the  tilt-wing,  fan-in-wing,  or  thrust- 
augmentation  concepts.  At  the  subsystem  level,  examples  are  the  advanced  turbine  engine 
gas  generator  (ATEGG)  and  aircralt  propulsion  system  integration  (APSI)  demonstrator 
programs  for  new  aircraft  turbine  engine  designs.  Examples  of  proof  of  performance  or 
cost  might  include  the  FlOO  engine  competitive  demonstration  and  the  APG-63  radar 
competitive  demonstration  prior  to  the  F-15  EMD  start.  Examples  of  proof  of  operational 
suitability  include  the  YA-10,  YF-16,  and  YAV-8B  tactical  aircraft,  where  the  emphasis 
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was  on  the  ability  of  the  aircraft  to  perforin  a  useful  operational  mission.  Operational 
prototypes  can  continue  to  be  useful  after  EMD  begins,  because  such  prototypes  can  be 
used  for  certain  testing  before  full-scale  development  test  articles  are  available. 

D.  CASES  EXAMINED 

We  cannot  evaluate  all  the  benefits  of  prototyping  in  a  quantitative  fashion.  The 
information  gained  through  prototyping  is  often  not  quantitatively  measurable.  In  addition, 
one  of  the  quantitative  benefits,  performance,  is  multi-dimensional  and  has  different 
dimensions  across  equipment  types.  If  a  program  proceeds  well  from  a  technical 
standpoint,  then  it  is  much  less  likely  to  encounter  schedule  and  cost  problems.  Cost  and 
schedule  problems  are  measurable.  By  measuring  planned  vs.  actual  schedules  and  costs, 
we  can  compare  program  outcomes  across  equipment  types. 

The  tactical  aircraft  cost-estimating  relationships  (CERs)  are  from  the  IDA  study  on 
tactical  aircraft  development  costs  [36],  and  the  munitions  CER  is  from  Yates,  Waller,  and 
Vaughn  [37].  There  are  some  limitations  in  the  data.  We  could  not  fully  identify  subsystem 
prototypes. 

E.  ANALYSIS 

Correlation  coefficients  between  cost  and  schedule  outcomes  and  a  prototyping 
dummy  variable  were  estimated.  They  were  always  negative  (lower  cost  growth  in  both 
phases  and  total,  lower  development  schedule  growth  and  lower  development  quantity 
growth  for  prototyped  systems),  but  not  statistically  significant. 

The  outcomes  for  prototyped  and  non-prototyped  programs  are  displayed  for  the 
total  database  and  for  types  of  equipment  with  more  than  five  prototypes  in  Table  V-1. 

1 .  Development  Cost  Growth 

This  is  one  area  in  which  aggregate  outcomes  differ  depending  on  whether  or  not 
the  observations  are  weighted.  For  the  total  database,  non-prototyped  systems  have  lower 
DCG  when  using  current  estimate  weights,  while  prototyped  systems  are  lower  when 
using  the  unweighted  data.  Among  tactical  munitions,  ECG  was  substantially  lower. 

2.  Unplanned  EMD  Articles 

Programs  with  prototyping  needed  only  5  percent  more  development  items  than 
planned,  while  non-prototyped  programs  needed  13  percent  more.  For  munitions 


V-5 


programs,  prototyped  programs  built  10  percent  fewer  EMD  articles  than  planned.  On  the 
basis  of  prototype  testing,  munitions  program  managers  actually  needed  fewer  EMD  test 
articles  than  planned.  This  benefit  is  not  seen,  however,  in  the  aircraft  and  die  ships. 

3.  Production  Cost  Growth  Lower 

Production  cost  growth  was  less  for  the  prototyped  systems  than  for  the  non- 
prototyped  systems  overall.  The  difference  was  greatest  for  the  tactical  munitions,  where 
weighted  production  cost  for  prototyped  programs  grew  by  56  percent,  while  that  for  non- 
prototyped  programs  grew  by  149  percent 

4.  Levels  of  Development  and  Production  Cost 

To  examine  the  effect  of  prototyping  on  the  levels  of  development  and  production 
costs,  we  turned  to  a  standard  tool  of  cost  analysis,  cost-estimating  relationships  (CERs). 
CERs  relate  technical  characteristics  of  a  weapon  system  to  its  development  or  production 
cost.  We  examined  the  residuals  of  the  CERs  to  determine  whether  there  was  any 
significant  difference  between  prototyped  and  non-prototyped  systems.  If  we  found  that 
prototyped  systems  had  significantly  higher  residuals,  this  would  indicate  that  a  system 
with  given  technical  characteristics  would  cost  more  if  it  were  prototyped.  Conversely,  if 
we  found  that  prototyped  systems  had  significantly  lower  residuals,  it  would  indicate  that 
prototyped  systems  generally  cost  less  than  non-prototyped  systems. 

We  were  able  to  perform  the  tests  for  three  equations:  a  tactical  aircraft  airframe  full- 
scale  development  CER,  a  tactical  aircraft  production  CER,  and  a  tactical  munitions  full- 
scale  development  CER.  For  tactical  aircraft  airframes,  there  is  no  significant  difference  in 
either  development  or  production  costs  that  could  be  explained  by  prototyping.  In  the  case 
of  tactical  munitions,  there  is  no  significant  difference  in  development  costs  between 
prototyped  and  non-prototyped  systems.  (We  were  unable  to  locate  a  sufficiently 
aggregated  CER  to  test  munitions  production  costs.)  Thus,  the  available  evidence  on  total 
costs  suggests  that  prototyped  systems  of  equivalent  technical  capability  do  not  cost 
significantly  more  or  less  than  non-prototyped  systems. 

5.  Schedule 

Overall,  prototyped  programs  took  2  years  longer  than  non-prototyped  programs 
from  Milestone  I  to  IOC  (significance  level  =  .06),  but  prototyping  made  no  difference  in 
the  time  from  Milestone  IJ  KX!!.  For  the  aircraft,  there  was  no  statistically  significant 
difference  in  either  interval.  Prototyped  aircraft  took  slightly  less  than  9  months  longer 
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from  Milestone  I  to  IOC  (117  vs.  108  months),  but  Milestone  II  to  IOC  times  were 
virtually  identical  (69.6  vs.  70.4  months).  The  prototyped  munitions  took  over  2  years 
longer  than  non-prototyped  munitions  (135  vs.  104  months),  but  the  difference  was  not 
statistically  significant.  (Moreover,  the  more  complicated  munitions  were  prototyped. 
When  we  control  for  this  relationship,  the  time  difference  decreases  to  1  year.)  The  length 
from  Milestone  II  to  IOC  was  actually  5  months  shorter  for  the  prototyped  munitions,  but 
again  was  not  statistically  significant. 

We  also  examined  more  detailed  evidence  on  schedule  for  a  set  of  nine  tactical 
aircraft  programs.  We  examined  these  programs  as  to  the  impact  that  prototyping  had  on 
schedule  length,  including  the  advanced  development  period,  FSD  start  to  first  flight 
period,  and  HMD  start  to  24th  unit  delivery  period.  The  F-16,  A- 10,  and  AV-8B  are  the 
prototype  programs,  and  the  F-4,  F-14,  F-15,  F/A-18,  F  ill,  and  S-3  are  the  non¬ 
prototype  programs.  (We  considered  the  F/A-18  a  non-prototype  program  because  of  the 
extensive  changes  in  the  Air  Force  prototype  design  that  resulted  in  the  Navy  HMD  design.) 

The  results  of  the  analyses  for  the  three  time  periods  are  presented  in  Table  V-2. 
The  first  equation  indicates  that  prototyping  increases  time  in  the  pre-FSD  period  by  19.33 
months,  more  than  doubling  it.  However,  time  to  first  flight  (in  the  second  equation)  is 
reduced  by  2.7  months.  Overall  F,SD  time  (third  equation)  is  around  1 1  percent  less  with 
prototyping.2 

Evidence  shows  that  the  prototyping  experience  reduces  time  in  FSD,  because  the 
prototype  can  be  used  early  in  FSD,  prior  to  the  availability  of  development  test  articles, 
helping  to  gain  information  early  in  the  program.  Thus,  the  cost  and  time  penalties 
associated  w'itn  prototyping  are  not  necessarily  as  large  as  might  be  assumed  by  simply 
adding  a  prototype  program  on  top  of  an  EMD  program.  Gaining  information  and  choosing 
attractive  options  while  precluding  unattractive  options  can  particularly  benefit  complex 
high-cost  programs.  The  evidence  from  examination  of  tactical  aircraft  schedules  bears  this 
out  with  an  1 1 -percent  reduction  of  FSD  time  (resulting  in  an  overall  schedule  increase  of 
15  percent  from  start  of  advanced  development  to  delivery  of  the  24th  aircraft). 


^  The  overall  FSD  equation  was  run  using  a  multiplicative  specificaUon: 

T24  =  bl*A  b2*e(COMPANY*b3)*e(PROTO*b4) 

In  order  to  interpret  the  results  more  easily,  the  parameters  associated  with  the  two  dummy  variables 
(b3  and  b4)  were  converted  from  ex;>onents  to  multipliers.  Thus,  prototyping  is  associated  with  an 
overall  FSD  time  tJiat  is  89  percent  of  the  FSD  time  without  prototyping. 
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Tabit  V-2.  EffMt  of  Prototyping 
on  Dovolopmant  Sotiodulo  Poriods 


Pre-FSD  Period 

PFSD  -  13.0  +19.33  (PROTO) 

(.0002)  (.0005) 

N  -  9;  «  .84;  Adjusted  R^  «  .81;  SEE  -  4.55 

Time  to  First  Flight  Period 

TFF  «  25.1  +  6.9  (COMPANY)  -  2.7  (PROTO)  +  2.9  (TEAM) 
(.0001)  (.002)  (.065)  (.060) 

N  *  9;  R^  =  .93;  Adjusted  R^  «  .89;  SEE  »  1.6 
Overall  FSD  Period 

T24  «  22.1  (A)-  X  1.15  (COMPANY)  x  .89  (PROTO) 
(.0011)  (.0.54)  (.035)  (.075) 

_ N  -  8:  >=  .95;  Adjusted  »  .91:  SEE  -  .05 _ 

Source;  Reference  [34], 

Notes;  Significance  levels  are  in  parentheses.  PROTO  »  dummy  variable  to 
indicate  whether  or  not  there  was  prototyping;  COMPANY  »  dununy  variable 
to  indicate  McDonnell  Douglas  as  prime  contractor;  and  TFAM  >  dummy 
variable  to  indicate  major  subcontractor/contractor  teaming  arrangements. 


Thus,  prototyping  may  take  some  additional  time.  This  time,  however,  must  be 
weighed  against  the  gains  in  cost  and  technical  predictability.  In  addition,  the  extra  time 
occurs  at  a  point  in  the  program  when  spending  rates  are  low  [35]. 

6.  Diverse  Strategies  Among  Weapon  Types 

The  two  equipment  groups  in  our  study  that  had  the  most  prototyping  were  aircraft 
and  tactical  munitions.  We  observed  very  different  strategies  regarding  the  prototyping  of 
these  two  groups.  Among  the  aircraft,  the  systems  pushing  the  state  of  the  art  the  least 
(such  as  the  F-5E  and  the  F-16)  were  prototyped,  while  others  that  were  more  technically 
difficult  (like  the  F-14)  were  not.  In  the  munitions,  the  opposite  occurred.  Systems  with  a 
high  level  of  technical  “reach”  like  Hellfire,  HARM,  and  Harpoon  were  prototyped. 

The  strategy  used  for  munitions  was  the  more  successful  of  the  two.  Munitions  are 
often  high-risk  programs  in  general.  They  are  less  glamorous  than  aircraft  and  therefore 
seem  to  get  less  management  attention.  Perhaps  the  building  and  testing  of  a  prototype 
serves  to  focus  attention  on  the  program.  In  any  event,  the  munitions  strategy  was  strongly 
successful.  We  would  expect  the  munitions  with  high  technical  reach  to  have  higher  cost 
growth  than  those  with  low  reach.  In  fact,  those  complicated  munitions  that  were 
prototyped  did  better  than  the  simple  ones  that  were  not  prototyped.  In  the  aircraft,  by 
contrast,  the  prototyping  strategy  did  not  seem  to  be  as  successful. 
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F.  FINDINGS  AND  RECOMMENDATIONS 


Prototyping  helps  developers  and  users  to  understand  the  technical  risks  and 
uncertainty  of  the  requirements.  The  quantitative  evidence  about  the  benefits  of  prototyping 
is  generally  positive  although  not  statistically  significant.  The  lower  development  cost 
growth  effect  is  particularly  pronounced  for  technically  challenging  programs. 
Development  quantity  growth,  the  need  to  build  unplanned  EMD  articles,  is  less  for  tactical 
munitions  programs.  The  benefits  of  prototyping  also  carry  over  into  production. 
Production  cost  growth  is  generally  less  for  prototyped  systems. 

These  benefits  come  with  some  increa.se  in  development  time.  However,  this 
additional  time  is  not  necessarily  very  long  (and  not  statistically  significant)  for  aircraft, 
and,  for  the  tactical  munitions,  it  may  be  more  related  to  technical  challenge  than  to 
prototyping.  Prototyping  is  a  leveraged  inve.stment.  The  government  is  buying  information 
relatively  cheaply,  early  in  the  program,  rather  than  discovering  problems  in  EMD  or 
production,  when  costs  (and  rates  of  expenditure)  are  higher.  Contrary  to  the  conventional 
wi.sdom,  our  evidence  suggests  that  prototyped  programs  do  not  cost  any  more  than  non- 
prototyped  programs. 

Prior  studies  of  prototyping  have  been  qualitative  and  have  emphasized  the 
uniqueness  of  each  acquisition.  Despite  this  uniqueness,  policymakers  should  use 
consistent,  clear  lessons  from  past  programs  to  set  strategy  for  new  programs.  The 
evidence  indicates  that  prototyping  is  a  successful  initiative  when  used  appropriately. 

Prototyping  enhances  the  credibility  of  major  programs,  particularly  given  the 
tendency  to  underestimate  technical  risk.  For  all-new  systems,  the  concept  demonstration 
phase  is  particularly  important.  Operational  suitability  prototyping  is  particularly  important 
in  times  of  budget  crunch,  since  we  are  particularly  eager  to  know  whether  a  system  will 
work  significantly  better  than  what  we  already  have. 

Prototyping  should  be  pursued  vigorously  where  significant  information  is  to  be 
gained  and  the  prototype  represents  only  a  small  percentage  of  acquisition  costs.  Given  the 
variability  of  these  findings,  guidelines  should  be  adopted  that  provide  bounds  to  costs  for 
the  benefits  to  be  expected  from  a  particular  application  of  prototyping. 

The  type  and  extent  of  prototyping  to  be  done  also  depends  on  the  nature  and  extent 
of  risk  in  the  program.  If  the  risk  is  largely  technical,  then  concept  and  design  prototyping 
are  the  most  important.  If  the  risk  is  that  requirements  are  uncertain,  then  proving  the 
technology  is  operationally  suitable  is  most  important.  If  there  are  concerns  about 
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production  costs  and  producibility,  it  may  be  necessary  to  add  a  test  of  operational 
suitability  with  production  article(s). 

As  rules  of  thumb  for  when  prototyping  makes  sense  relative  to  its  likely  payoff, 
we  suggest  that  the  prototype  cost  should  be  less  than  25  percent  of  the  EMD  cost  estimate, 
10  percent  of  the  acquisition  cost  estimate  (EMD  and  procurement),  or  5  percent  of  the  life- 
cycle  cost  estimate,  whichever  is  highest 

These  rules  of  thumb  can  be  adapted  for  technical  risk  and  schedule  criticality.  If 
technical  risk  is  high,  then  the  cost  estimates  upon  which  these  rules  of  thumb  are  based 
have  considerable  risk  attached  to  them.  For  example,  if  technical  risk  is  low,  schedule  is 
critical,  and  a  prototype  would  cost  20  percent  of  EMD  cost,  then  it  would  not  make  sense 
to  undertake  one.  On  the  other  hand,  if  technical  risk  is  known  to  be  very  high,  .schedule  is 
not  critical,  and  a  prototype  would  cost  30  percent  of  the  EMD  cost  estimate,  then 
prototyping  makes  sense. 

Our  quantitative  analysis  was  not  extensive  enough  to  support  development  of  a 
cost/benefit  model  for  prototyping.  Nevertheless,  we  have  taken  some  important  first  steps 
toward  such  a  model.  A  key  element  of  such  a  model  is  a  better  measure  of  technical  risk 
early  in  the  acquisition  process. 

It  would  be  useful  for  the  government  to  capture  the  costs  of  prototypes  to  refine 
EMD  and  procurement  cost  estimates.  The  literature  on  this  subject  is  surprisingly  sparse. 

It  would  also  be  useful  to  study  the  impact  of  prototyping  in  combination  with  otlier 
initiatives  such  as  design-to-cost  and  contract  incentives.  In  addition,  the  impact  of  a 
generalized  strategy  of  prototyping  across  programs  should  be  assessed.  This  should 
include  its  effect  on  competition  and  on  the  ability  of  industry  to  develop  and  produce  new, 
technologically  sophisticated  weapon  systems. 
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VI.  CONTRACT  INCENTIVES 


A.  BACKGROUND 

Incentive  contracting  has  a  long  history  in  acquisition.  The  early  1960s  was  an 
“incentive  era”  in  which  the  government  attempted  to  reduce  costs  through  increased  use  of 
firm-fixed-price  and  incentive-type  contracts  [38].  Two  general  types  of  incentive  contracts 
are  in  use  today— cost  plus  (or  fixed-price)  incentive  fee  and  cost  plus  (or  fixed-price) 
award  fee. 

In  fixed-price-incentive  contracts,  the  contract  has  a  target  and  a  ceiling  price.  If  the 
contractor  meets  the  target  price,  it  receives  the  full  incentive  fee.  If  it  goes  over  the  target 
price,  costs  are  split  with  the  government  according  to  a  sharing  formula  (50/50,  60/40, 
etc.)  up  to  the  ceiling  price.  Costs  above  the  ceiling  price  are  covered  by  the  contractor. 
Incentive  contracts  can  also  be  written  to  include  incentives  for  system  performance  or 
delivery  schedule.  However,  the  most  prevalent  reason  for  incentive  contracts  is  to  share 
cost  risks  with  the  contractor.  The  government  wants  the  contractor  to  produce  efficiently 
and  at  the  lowest  cost. 

Fixed-price  award  fee  contracts  have  widely  differing  structures.  Under  these 
contracts,  the  fee  is  awarded  based  upon  performance  of  goals  set  out  in  the  contract. 
Award  fee  contracts  are  more  flexible  than  incentive  fee  contracts.  They  can  incorporate  a 
variety  of  goals,  including  non-cost  goals  such  as  delivery  dates  or  reliability  and 
maintainability  goals.  Weights  may  be  given  to  individual  goals.  A  review  board  may  be 
appointed  to  determine  how  much  of  the  fee  is  to  be  awarded. 

The  General  Accounting  Office  (GAO)  reviewed  62  fixed-price  incentive  contracts 
from  1977-84  to  determine  how  the  final  price  of  each  compared  with  the  contract’s 
established  target  and  ceiling  prices  [39].  Fifty-six  of  these  contracts  were  for  over  $1 
million  and  22  were  for  over  $10  million  each.  GAO  analysts  expected  to  find  a  clustering 
of  final  prices  very  close  to  the  target  price  and  an  increasing  tendency  for  final  prices  to  be 
lower  than  the  target  price  (or  for  lower  overruns)  as  contractor  sharing  ratios  increased. 
They  found  that  the  final  prices  on  58  percent  of  the  contracts  were  within  5  percent  of  the 
target,  and  92  percent  were  within  10  percent  of  the  target.  However,  GAO’s  findings  and 
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other  research  findings  cited  in  the  report  were  that  final  contract  costs  and  price  seem 
unrelated  to  the  sharing  ratio. 

B.  BENEFITS  AND  WEAKNESSES 

Benefits  of  conuact  incentives  to  the  government  include: 

•  Cost  savings  compared  with  what  costs  would  be  in  fixed-fee  contracts 

•  The  degree  to  which  the  contractors  behave  in  the  way  the  government  wants. 

Advocates  of  incentive  contracts  would  say  that  they  are  a  good  substitute  for 
competition,  which  is  very  difficult  to  establish  in  major  weapons  systems  liecause  of  the 
small  size  of  the  industry.  Incentive  contracts  are  an  attempt  to  create  market-like  signals  in 
an  industry  in  which  competition  is  difficult.  The  rationale  is  that  the  use  of  incentive  fees 
and  award  fees  is  a  low-cost,  high-leverage  proposition.  Unlike  some  of  the  other 
initiatives  we  have  examined,  such  as  prototyping  and  competition,  incentive  contracting 
does  not  involve  substantial  up-front  costs. 

Weaknesses  of  incentive  contracts  are  the  additional  costs  to  the  government, 
including  the  following: 

•  The  extra  costs  of  incentive  and  award  fees  over  and  above  those  that  would  be 
given  on  cost-  plus  contracts  and 

•  The  extra  cost  of  administering  incentive  contracts,  as  opposed  to  cost-plus 
contracts.  (Award  fee  contracts  may  have  higher  administrative  costs  than 
incentive  fee  contracts,  because  of  the  necessity  of  measuring  and  monitoring 
contractor  performance.) 

Another  possible  pitfall  is  that  cross-program  effects  may  dominate  and  increase 
costs.  For  example,  suppose  that  a  contractor  is  working  on  two  programs  simultaneously 
in  one  plant.  Program  A  is  sole  source,  while  Program  B  is  competitive.  Even  if  the 
contract  for  Program  A  has  an  incentive  clause,  it  may  make  sense  for  the  contractor  to 
define  costs  in  a  way  that  increases  the  cost  of  Program  A  and  keeps  the  cost  of  Program  B 
low,  in  order  to  keep  the  competitive  Program  B. 

C.  ANALYSIS 

Our  database  had  67  programs  with  incentives  of  any  type  in  EMD  and  56  with 
incentives  in  production.  In  most  cases,  the  information  we  had  was  an  indication  of 
whether  or  not  incentives  were  used  rather  than  information  on  types,  sharing  ratios,  and 
provisions  of  individual  contracts.  Figures  VI- 1  and  VI-2  show  outcomes  for  programs 
with  development  .ontract  incentives  and  production  .  ontfact  incentives,  rtspectively. 


VI-2 


(67)  (31)  (25)  (10)  (19)  (8)  (7)  (2)  (2)  (1)  (14)  (10) 

All  Tactical  Aircraft  Ships  Vahiclas  Other 
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Notes;  Numbers  in  parentheses  are  numbers  of  programs.  Difference  was  statistically 
significant  for  the  following:  all  programs,  ships,  and  other. 

Figure  Vl>1.  Development  Cost  Growth  for  Programs 
With  and  Without  Incentives  in  FSD 

Figure  VI- 1  indicates  that  development  cost  growth  is  lower  in  programs  with 
incentive  conu  acts.  Thi.s  is  true  for  all  of  the  equipment  groups  we  examined  except  for  the 
vehicle  programs,  which  are  few  and  have  displayed  anomalous  results  in  several  areas. 
Some  of  these  differences  were  statistically  significant:  ships  (.02),  other  (.01),  and  overall 
(.09). 

In  production,  the  results  are  mixed.  Contract  incentives  in  production  seem  to  have 
the  biggest  cost  growth  reduction  effect  in  the  “other”  category  and  in  the  vehicles.  For  all 
of  the  other  equipment  groups,  cost  growth  is  higher  for  programs  with  contract  incentives. 
None  of  the  differences  were  statistically  significant. 

We  make  two  observations  about  these  results.  First,  our  measurement  was  very 
simple.  We  included  as  incentive  contracts  any  type  of  incentive  contract  awarded  to  a 
prime  contractor  at  any  time.  In  HMD,  there  are  relatively  few  contracts,  so  this  measure  is 
fairly  pure.  In  production,  however,  it  is  typical  to  produce  the  first  few  lots  under  cost- 
plus  contracts,  then  to  move  to  fixed-price  incentive  contracts  when  production  procedures 
are  well-established.  Thus,  57  of  our  programs  have  incentives  in  production,  while  only 
25  00  not.  More  work  is  needed  on  measurement  issues.  Second,  the  results  are  intuitively 
appealing.  Incentive  contracts  appear  to  have  positive  effects  in  development.  In 


production,  costs  tend  to  be  influenced  by  many  factors  other  than  contract  type,  such  as 
stretch.  Programs  without  production  incentives  tended  to  be  stretched  more  than  programs 
with  incentives.  However,  the  production  result  is  also  consistent  with  the  possibility  of 
perverse  cross-program  effects.  Overall,  these  results  indicate  the  potential  for  digging 
deeper. 


(56)  (26)  '  (23)  (7)  '  (15)  (10)  '  (1)  (2)  '  (7)  (7) 

All  Tactical  Aircraft  Vehicles  Other 


Programs  Munitions 

■  With  Incentives  H]  Without  Incentives 

Notes:  Numbers  in  parentheses  are  numbers  of  programs.  All  ships  had  production 
incontive  contracts,  and  their  mean  cost  growth  was  1 1  percent. 

Figure  VI-2.  Production  Cost  Growth  for  Programs 
With  and  Without  Incentives  in  Production 

IDA  case  .studies,  program  office  visits,  and  industry  surveys  from  prior  work  [1] 
indicated  that  incentives  were  successful.  Both  government  and  industry  believed  incentive 
contracts  to  be  effective,  although  we  did  not  independently  evaluate  savings  estimates.  In 
general,  government  representatives  believed  that  they  received  what  they  wanted  in  cases 
of  award  fee.  Industry  representatives  pointed  out  that  incentive  and  award  fees,  which 
might  seem  small  as  a  percentage  of  the  total  contract,  amount  to  a  large  percentage  of  total 
allowable  profit.  Managers’  bonuses  might  be  tied  to  their  performance  in  obtaining  full 
incentive  or  award  fees. 
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D.  FINDINGS  AND  RECOMMENDATIONS 


IDA  found  that  development  cost  growth  was  lower  for  programs  with  incentive 
contracts  than  for  programs  without  them.  In  production,  results  were  mixed. 

Contract  incentives  are  a  madity  available,  cheap  way  of  potentially  reducing  costs, 
without  imposing  new  regulatory  burdens  and  without  the  need  for  new  legislative 
authority. 

Our  findings  on  incentive  contracts  need  to  be  considered  in  the  context  of  two 
initiatives  with  similar  cost-reducing  objectives — competition  and  fixed-price  development. 
The  acquisition  environment  of  the  1990s  may  not  be  all  that  hospitable  to  competition, 
because  the  number  of  potential  competitors  will  be  declining.  In  addition,  because  of  the 
decline  in  the  number  of  programs  going  into  production,  the  old  strategy  of 
underestimating  development  and  “getting  rich”  in  production  will  be  less  available.  In  this 
environment,  are  incentive  contracts  a  good  substitute  for  other,  less  piactical  initiatives? 
With  respect  to  competition,  incentive  contracts  appear  to  be  administratively  simpler, 
although  the  potential  saving.*;  may  be  lower.  With  respect  to  fixed-price  development, 
incentive  contracts  appear  to  be  more  successful. 

In  the  new  environment,  it  seems  worthwhile  to  have  incentives  available  as  an 
option.  IDA  macro  and  micro  analyses  pointed  to  the  same  conclusion:  contract  incentives 
work,  at  least  in  development.  Moreover,  contract  incentives  are  fairly  simple  and 
inexpensive  to  implement.  However,  much  more  information  is  needed  on  how  to  design 
and  time  incentives  so  that  they  work  best. 

Based  on  our  aiialysis,  we  recommend: 

*  Wherever  possible,  incentive  contracts  be  used  in  development,  and 

•  Measures  of  contract  incentives  be  refined  to  include  exact  contract  type  and, 
for  production,  the  lot  numbers  to  which  incentive  contracts  were  applied. 
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VII.  MULTI-YEAR  PROCUREMENT 


Multi-year  procurement  (MYP)  describes  the  acquisition  situation  where  DoD 
contracts  for  more  than  the  current  year’s  requirement.  DoD’s  planned  requirements,  for  up 
to  a  five-year  period,  arc  acquired  without  having  total  funds  available  at  the  time  of 
contract  award.  Thus,  an  MYP  contract  is  an  alternative  to  a  series  of  annual  contracts  in 
which  the  end  items  are  procured  one  year  at  a  time.  Through  economic  quantity  buys, 
MYP  is  expected  to  reduce  the  cost  of  procuring  a  weapon  system. 

A.  BACKGROUND 

The  U.S.  Army  began  testing  the  concept  of  MYP  on  small  automotive  motors  in 
the  early  1960s.  DoD  actively  used  MYP  for  weapon  systems  acquisition  throughout  the 
1960s,  During  this  period,  weapon  programs  were  typically  funded  with  no-year  (no  two- 
or  three-year  obligation  requirement)  funds,  and  no  special  authorization  was  required  to 
award  contracts  on  a  multi-year  basis.  DoD  claimed  cost  savings  and  a  high  degree  of 
program  stability  [40], 

In  1972,  the  Navy  canceled  a  pair  of  shipbuilding  contracts  incurring  cancellation 
charges  of  $388  million.  Although  the  problems  with  these  particular  contracts  were  not 
necessarily  related  to  the  fact  that  they  were  multi-year,  Congress  nonetheless  was  not 
pleased  with  such  a  large  unfunded  liability.  To  prevent  a  recurrence  of  these  unexpected 
cancellation  payments  on  multi-year  contracts.  Congress  established  a  maximum 
cancellation  ceiling  of  $5  million  in  the  FY  1973  Defense  Authorization  Act.  Contractors 
refused  to  accept  multi-year  contracts  for  major  systems  acquisitions  with  only  $5  million 
cancellation  ceiling.  If  a  major  program  were  canceled  after  the  first  year,  the  contractor 
would  face  significant  unrecovered  costs.  For  the  remainder  of  the  197()s,  the  effect  of  the 
limit  on  cancellation  ceilings  was  to  virtually  eliminate  the  use  of  MYP  on  major  systems 
acquisition  [40]. 

A  Defense  Science  Board  (DSB)  study  rekindled  interest  in  MYP  by  estimating  the 
DoD  could  save  10  to  15  percent  of  program  costs  by  using  MYP  on  major  programs  [  14]. 
DoD  endorsed  the  DSB  position,  and  Deputy  Secretary  of  Defense  Frank  Carlucci  adopted 
expanded  use  of  multi-year  procurement  as  one  of  the  “Carlucci  initiatives." 
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Congress  passed  legislation  in  the  FY  1982  Authorization  Bill,  Public  Law  97-86, 
set  out  guidelines  for  MYP.  Public  Law  97-86  authorized: 

•  Cancellation  ceilings  of  $  100  million  without  notifying  Congress.  Thirty  days 
notice  to  Congress  was  required  for  contracts  with  ceilings  in  excess  of  $100 
million.  Ceilings  could  either  be  funded,  or  left  unfunded  and  carried  as  a 
contingent  liability. 

•  Use  of  MYP  with  annual  funds  for  supplies  and  services. 

•  Broadened  coverage  of  the  cancellation  ceiling  to  include  recurring  costs  (costs 
of  out-year  components,  parts,  and  work  in  process),  as  well  as  previously 
allowed  non-recurring  costs  and  economic  lot  buys. 

•  Advanced  buys,  both  in  the  case  of  long-lead-time  items  and  economic  order 
quantities,  for  more  than  one  year  beyond  the  current  year’s  requirements. 

•  MYP  contracts  to  cover  up  to  a  five-year  period. 

In  Public  Law  97-86,  criteria  were  established  that  multi-year  candidates  must  meet 
with  congressional  approval  prior  to  authorization  of  funding.  In  order  for  MYP  to  be  of 
benefit  to  the  government,  the  estimated  cost  savings  have  to  be  significant  because  multi¬ 
year  contracting  can  reduce  future  budget  flexibility.  Whether  savings  are  enough  to  offset 
the  risks  imposed  by  reduced  budget  flexibility  is  judged  by  Congress.  In  the  past. 
Congress  has  asked  the  General  Accounting  Office  to  make  this  assessment  [41  through 
43].  To  do  this,  program  risk  in  the  following  areas  is  assessed; 

•  Confidence  in  the  cost  estimate, 

•  Requirement  stability, 

•  Funding  stability,  and 

•  Configuration  or  design  stability. 

Confidence  in  the  cost  estimate  requires  that  the  contract  cost  estimates  and  the 
anticipated  cost  savings  be  realistic.  Cost  savings  are  figured  as  the  difference  between  cost 
estimates,  proposals,  or  negotiated  prices  for  the  multi-year  contract  and  the  cost  of 
procuring  the  same  quantities  in  the  same  time  periods  with  successive  annual  contracts. 
The  services  generally  use  proposals  or  negotiated  contracts  from  the  applicable  contractor 
on  both  an  annual  and  multi-year  basis,  and  then  compare  and  analyze  those  proposals  to 
estimate  savings  from  the  MYP  approach. 

A  stable  requirement  means  the  total  quantity  and  procurement  rate  will  not  vary 
substantially  (principally  avoiding  downward  adjustments)  over  the  term  of  the  multi-year 
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contract.  Decreases  in  the  procurement  quantities  can  cause  termination  of  the  multi-year 
contract  and  create  unit  cost  increases,  which  could  reduce  savings. 

Both  DoD  and  the  Congress  must  be  committed  to  ensuring  that  sufficient  funds  are 
provided  to  complete  the  multi-year  contract  at  planned  production  rates.  A  turbulent 
funding  history  for  a  weapon  system  may  suggest  an  unstable  requirement,  a  relatively  low 
funding  priority,  or  wavering  support;  this  may  make  the  system  inappropriate  for  multi¬ 
year  contracting.  Disagreements  among  the  military  services,  OSD,  and  the  Congress 
concerning  the  appropriate  production  rate  and  required  funding  for  a  system  are  often 
signals  that  funding  is  not  stable. 

Test  and  evaluation  should  be  complete  and  demonstrate  that  the  system,  and 
therefore  the  design,  is  operationally  effective.  The  Senate  Committee  on  Appropriations, 
has  always  recommended  that  the  multi-year  approach  be  reserved  for  established 
production  operations  and  state-of-the-art  technology.  Moreover,  a  program  should  be 
Judged  mature  and  stable  only  after  research  and  development  and  one  or  two  production 
runs  have  been  completed  successfully. 

After  passage  of  Public  Law  97-86,  the  DoD  immediately  claimed  savings  of  $325 
million  by  using  MYP  for  FY  1982  weapon  system  programs.  In  1983,  the  Grace 
Commission  advocated  greater  use  of  MYP  [10]  and  staled  that  DoD  might  save  as  much  as 
$3  billion  over  the  next  several  years  with  more  aggressive  use  of  MYP  [40  and  44]. 
However,  in  1983  Congress  placed  limits  on  the  use  of  MYP.  Advance  congressional 
notification  of  all  MYP  programs  with  cancellation  ceilings  over  $20  million,  rather  than 
over  $1(K)  million,  was  required.  Congressional  notification  of  all  economic  order  quantity 
purchases  was  required.  Congress  also  imposed  a  requirement  that  all  four  Defense  Budget 
committees  be  notified  of  programs  selected  as  MYP  candidates. 

To  provide  greater  assurance  of  the  validity  of  estimated  savings,  the  Congress  has 
mandated  a  two-step  multi-year  approval  process;  Proposed  multi-year  contract  costs  are 
provided  both  with  the  budget  submission  and  again  just  before  contract  award.  Defense 
Appropriation  Acts  since  FY  1984  have  included  language  that  reserves  final  multi-year 
approval  until  negotiated  contract  prices  are  submitted  to  the  House  and  Senate  Armed 
Services  and  Appropriations  Committees  at  least  .30  days  before  the  contract  award.  This 
allow.s  the  committees  to  compare  the  estimates  presented  in  the  justification  packages  with 
the  actual  proposed  contract  amounts. 
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In  1990,  GAO  examined  6  programs  that  had  been  approved  for  multi-year 
procurement  by  OSD  and  the  Congress,  to  determine  the  extent  to  which  the  MYP  criteria 
had  actually  been  satisfied  [451.  All  six  programs  failed  one  or  more  of  the  criteria. 

B.  BENEFITS  AND  WEAKNESSES 

In  its  fiscal  years  1982  through  1989  budget  requests,  OSD  proposed  at  least  60 
candidates  to  Congress  for  approval  of  multi-year  procurement  authority.  OSD  estimated  a 
total  potential  savings  for  these  programs  of  approximately  $13.4  billion  then-year  dollars. 
Total  procurement  value  of  the  rnulti-year  candidates  was  estimated  as  $78.8  billion  with 
rnulti-year  contracting.  The  information  was  accumulated  from  the  FY  1982  through  FY 
1989  OSD  “Justification  of  Estimates  for  Multi-Year  ProcuremeiiL” 

Typically,  the  majority  of  savings  in  a  multi-year  arrangement  comes  from  the 
ability  to  procure  vendor  items  more  economically  on  a  multi-year  basis  than  on  an  annual 
basis.  Manufacturing  savings,  slightly  more  than  a  third  of  the  planned  total,  are  achieved 
by  increased  prime-contractor  manufacturing  efficiencies  made  possible  by  stable 
production  rates  and  the  inc.  eased  length  of  production. 

Multi-year  justification  packages  submitted  to  Congress  often  include  estimates  of 
industrial  base  enhancements  that  would  result  by  applying  the  multi-year  approach  to  any 
candidate  weapon  system.  Examples  of  anticipated  enhancements  cited  by  the  services 
include: 

•  Enhanced  investment  in  infrastructure  at  the  prime  contractor  and  vendor 
levels. 

•  Enhanced  training  programs, 

•  Improved  vendor  skill  levels,  and 

•  Improved  competition  at  vendor  levels. 

These  enhancements  then  translate  into  increased  production  capacities  and  increased 
effectiveness. 

The  stability  in  contractor  and  subcontractor  operation  associated  with  multi-year 
contracts  can  create  a  level  of  business  certainty  more  conducive  to  enhancing  the  industrial 
base  than  annual  procurement.  Nevertheless,  it  is  difficult,  if  not  impossible,  to  identify  the 
industrial  base  enhancements  that  have  occurred  as  a  result  of  a  multi-year  contract  that 
would  not  have  occurred  under  annual  contracts.  No  attempt  to  quantify  a  value  for 
industrial  base  enhancements  was  applied  during  this  analysis. 
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In  spite  of  the  potential  benefits,  several  weaknesses  of  MYP  must  be  considered. 

They  are: 

•  Multi-year  contracting  can  reduce  future  budget  flexibility  for  DoD  and  the 
Congress.  This  is  especially  true  in  times  of  budget  uncertainty  and  declining 
budgets.  If  budget  conditions  are  expected  to  be  unpredictable  (or  general 
funding  to  be  unstable)  during  the  timeframe  of  a  proposed  multi-year  conU'act, 
and  if  changes  are  forced  on  the  MYP  program,  OSD  has  two  options:  (1) 
renegotiate  the  MYP  contract  or  (2)  cancel  the  contract.  In  either  case,  the  result 
is  likely  to  be  more  costly  than  a  series  of  annual  contracts  over  the  same  time 
period  would  be. 

•  Multi-year  contracting  requires  a  substantial  amount  of  up-front  funding.  The 
government  will  incur  higher  borrowing  costs  associated  with  accelerated 
expenditures  under  multi-year  contracting.  Cost  savings  must  offset  these 
additional  government  borrowing  costs. 

•  Multi-year  procurement  contracts  often  specify  a  contractor  cancellation  fee.  In 
order  for  MYP  to  work,  the  contractor  must  feel  protected  enough  to  procure 
from  vendors  at  economic  rates.  The  cancellation  fees  ensure  that  if  the  contract 
is  terminated,  the  contractor  and  vendors  to  the  prime  contractor  will  not  go 
entirely  uncompensated  for  procurement  of  parts  or  materials  greater  than 
would  have  been  procured  in  an  annual  contracting  environment. 

•  MYP  can  result  in  loss  of  design  flexibility.  Unanticipated  changes  in  the  threat 
and  incorporation  of  rapid  changes  in  technology  cannot  be  easily  addressed  in 
the  multi-year  environment.  Renegotiation  of  the  multi-year  contract  is 
generally  required  Even  so,  changes  may  be  more  difficult  to  incorporate  Uian 
under  an  annual  contract.  This  is  btrcause  the  prime  contractor  may  produce 
heavily  in  the  early  years  of  the  multi-year  contract  and  may  have  tooled 
accordingly.  At  the  very  least,  the  prime  contractor  will  have  made 
commitments  with  vendors  for  materials,  specified  in  the  earlier  design,  that 
would  make  an  immediate  shift  to  the  enhanced  design  costly. 

•  Use  of  MYP  may  also  reduce  the  funding  available  for  other  acquisition 
programs.  The  full-funding  requirements  used  under  MYP  can  result  in  the 
crowding  out  of  other  programs.  The  services’  flexibility  in  assigning  priorities 
among  various  programs,  and  to  reallocate  funding  among  the  programs,  is 
reduced  by  MYP  [46  and  47]. 

Primarily  because  of  the  funding  comniiunents  discussed  in  this  section,  neither  the 
services  nor  the  Congress  have  been  willing  to  commit  many  programs  to  multi-yea*' 
funding.  Historically,  fever  than  half  of  the  candidates  proposed  in  any  fiscal  year  are 
approved  for  multi-year  funding. 
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C.  ANALYSIS 


We  examined  the  programs  in  the  IDA  database  that  pursued  a  multi-year 
ac(juisition  strategy  to  show  whether  or  not  MYP  had  indeed  contributed  to  production  and 
total  program  cost  savings.  We  examined  how  MYP  contributed  to  "cost  savings”  in  the 
sense  that  these  programs  experienced  significantly  lower  production  and  total  program 
cost  growth  than  other  programs  in  the  IDA  data  population.  (MYP  may  contribute  to  the 
avoidance  of  cost  growth  experienced  by  programs  pursuing  other  acquisition  strategies.) 

We  assigned  programs  with  production  data  available  to  one  of  four  groups — 
mature  MYP  programs  (those  with  three  or  more  years  of  MYP  experience),  immature 
MYP  programs  (those  with  current  MYP  contracts,  but  with  less  than  three  years  of 
experience),  MYP  candidates  (those  considered  but  not  approved),  and  programs  that  were 
not  official  candidates  for  MYP.  (Later  versions  of  the  Bradley  FVS  and  MLRS  had  MYP 
contracts,  but,  since  we  examined  the  first  version,  we  considered  then  to  be  non-MYP 
programs.) 

The  outcomes  of  the  mature  MYP  programs  are  shown  in  Table  VIM.  Table  VII-2 
compares  outcomes  among  the  four  classes  of  programs. 


Tabla  Vli-1.  Outcomes  for  Mature  MYP  Programs  (Percent) 


Procram 

PCG 

TPCG 

PSG 

Stretch 

B-IB 

4 

-1 

-17 

0 

-17 

F-16 

21 

21 

351 

361 

-2 

CH-47D 

34 

32 

49 

31 

14 

UH-60A/L 

28 

23 

129 

104 

12 

TOW 

78 

70 

127 

41 

285 

DM3P 

22 

15 

29 

13 

14 

NAVSTAR  GPS 

24 

11 

71 

93 

-11 

DSP 

0 

10 

25 

40 

-11 

DSCS  III 

70 

74 

-13 

8 

-19 

Shillelagh 

.‘>4 

47 

44 

-11 

62 

Patriot 

78 

68 

15 

-.54 

161 

MLRS 

-12 

-8 

72 

66 

4 

M198  Howitzer 

29 

31 

-9 

-11 

2 

SURTASS/T-AGOS 

63 

68 

54 

50 

3 

SSN-688 

-1 

-1 

60 

520 

-74 

Production  outcome  measures  are  considerably  better  for  the  mature  MYP  programs 
than  for  the  non-MYP  programs.  Production  cost  growth  is  only  24  percent,  vs.  69  percent 
for  the  non-MYP  programs.  MYP  programs  also  tend  to  increase  their  planned  production 
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quantity  and  to  be  stretched  considerably  less.  Production  quantity  growth  is  78  percent  for 
the  mature  MYP  programs,  vs,  only  23  percent  for  the  non-MYP  programs.  Mature  MYP 
programs  tend  to  be  suetched  less — 27  percent  vs.  81  percent. 

Table  VII-2.  Outcomaa  for  Programs  With  and  Without  MYP  (Parcent) 


Outcome 

Measure 

Mature 

MYP 

(N«15) 

Immature 

MYP 

(N--S)  - 

MYP 

Candidates 

(N-4) 

Without 

MYP 

(N«58) 

DCG 

27 

30 

10 

33 

PCG 

24 

52 

36 

69 

TPCG 

23 

45 

29 

55 

PSG 

65 

67 

113 

49 

PQG 

78 

14 

54 

23 

Stretch 

27 

57 

40 

81 

Note:  Co.^!  growth  figures  are  weighted  by  the  current  estimate. 


Critics  of  MYP  argue  that  MYP  uses  a  “cream-skimming”  strategy — programs  that 
receive  authorization  for  MYP  would  have  come  out  well  anyway.  In  order  to  shed  some 
light  on  this  issue,  we  examined  the  outcomes  of  the  MYP  candidate  programs.  These 
programs  were  viewed  by  the  DoD  as  meeting  the  criteria  for  MYP,  but  they  were  not 
approved  by  Congress.  If  the  candidates  had  outcomes  as  favorable  as  the  MYP  programs, 
this  would  indicate  that  MYP  has  no  independent  effect. 

The  results  of  the  MYP  candidate  programs  are  in-between — they  exhibit  better 
outcomes  than  the  non-MYP  programs,  but  not  as  good  as  the  outcomes  for  the  mature 
MYP  programs.  The  difference  in  production  cost  growth,  the  key  measure,  is  not 
statistically  significant. 

Surprisingly,  the  immature  MYP  programs  exhibit  higher  production  cost  growth 
than  the  MYP  candidates.  Among  these  programs  are  Improved  Hawk  (207  percent 
production  cost  growth),  AH-64A  (85  percent),  and  HR  Maverick  (55  percent). 

By  its  very  nature,  multi-year  contracting  enhances  program  stability,  which  also 
contributes  to  lower  cost  grov'th.  It  is  difficult  to  separate  the  effects  of  these  two  factors. 
However,  the  MYP  candidate  programs  received  roughly  the  same  level  of  production 
stability  (40  percent  stretch  vs.  27  percent  for  the  mature  MYP  programs)  and  had  higher 
cost  growth. 
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D.  FINDINGS  AND  RECOMMENDATIONS 


The  principul  objective  of  multi-year  contracting  is  to  reduce  procurement  costs. 
OSD  justifications  indicate  that  the  services  expected  to  obtain  savings  of  10  to  20  percent 
when  multi-year  conuacting  is  used.  We  were  unable  to  examine  cost  savings  rigorously. 
We  were,  however,  able  to  observe,  considerably  lower  cost  growth  for  multi-year 
programs  as  opposed  to  non-multi-year  programs. 

At  least  some  of  the  favorable  outcomes  observed  are  probably  due  to  the  criteria 
that  are  applied  to  multi-year  candidates — stable  system  requirement,  system  design, 
program  funding,  and  because  of  this  program  and  stability,  confidence  in  the  production 
cost  estimate — rather  than  to  the  implementation  of  the  multi-year  contracting  strategy. 

The  fifteen  programs  that  have  employed  multi-year  contracts  for  at  least  three  years 
exhibit  lower  production  and  total  program  cost  growth  than  do  the  general  population  of 
programs  examined  during  this  study.  Programs  that  have  MYP  tend  to  be  stretched  out 
considerably  less  than  non-MYP  programs.  In  part,  this  is  because  the  government  wants 
to  protect  programs  for  which  multi-year  commitments  have  been  made  and  to  avoid  paying 
cancellation  fees.  This  added  program  stability  certainly  contributes  to  the  low  cost  growUi 
observed.  However,  we  also  observe  that  MYP  candidate  programs  had  about  the  same 
level  of  stability,  but  had  36  percent  average  production  cost  growth,  vs.  24  percent  for  the 
mature  MYP  programs.  This  indicates  that  MYP  is  having  some  independent  favorable 
effect. 

We  recommend  that  OSD  continue  support  for  multi-year  procurement  candidates. 
The  office  should  continue  use  of  the  present  guidelines  that  call  for  evaluation  of  stability 
of  the  requirement,  the  system  design,  the  funding  plan,  and  realism  of  the  cost  estimate. 
Our  examples  indicate  that  well-managed,  stable  programs  can  indeed  benefit  from  MYP. 
However,  applying  MYP  to  all  programs  is  impractical. 
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VIII.  DESIGN-TO-COST 


A.  BACKGROUND 

The  design-to-cost  (DTC)  concept  was  instituted  as  one  of  several  reforms  to  DoD 

procurement  practices.  Developed  primarily  by  former  Deputy  Secretary  of  Defense  David 

Packard  and  by  former  Director  of  Defense  Research  and  Engineering  (DDR&E)  John 

Foster,  the  purpose  of  DTC  was  to  develop  a  unit  cost  goal  early  in  the  design  process  and 

to  design  to  that  goal.  DoD  Directive  424S.3  of  6  April  1983  defines  DTC  as: 

. .  .an  acquisition  management  technique  to  achieve  defense  system  designs  that  meet  staled 
cost  requirements.  Cost  is  addressed  on  a  continuing  basis  as  part  of  a  system’s 
development  and  production  process.  The  technique  embodies  early  establishment  of 
realistic  but  rigorous  cost  goals,  and  thresholds  and  a  detennined  effort  to  achieve  them. 

The  DTC  goal  is  initially  expressed  in  terms  of  the  average  unit  flyaway  (or 
rollaway  or  sailaway)  cost  associated  with  an  end  item  of  military  hardware.  As  the  ability 
to  translate  operations  and  support  cost  elements  into  “design  to”  requirements  improves, 
DTC  goals  and  thresholds  are  related  to  total  life-cycle  cost  (LCC). 

On  13  July  1971,  DTC  became  official  policy  in  DoD  Directive  5{X)0.1.  The 
directive  provides  that  system  development  be  “continuously  evaluated  against  these 
(design-to)  requirements  with  the  same  vigor  as  that  applied  to  technical  requirements.”  On 
18  June  1973,  Deputy  Secretary  of  Defense  Clements  issued  a  memorandum  entitled 
“Design  to  a  Cost  Objective  on  DSARC  Programs,”  directing  Lhat  a  DTC  goal  be  applied  to 
all  major  DSARC  programs.  At  this  point  the  concept  moved  from  being  a  goal  (in  DoD 
Directive  5900.1)  to  being  a  requirement  for  major  programs  in  the  acquisition  process.  In 
October  1973,  two  documents  on  methodology  for  DTC  were  released:  (1)  “Joint  Design  to 
Cost  Guide,”  dated  3  October  1973  and  (2)  “Cost  to  Produce  Handbook,”  dated  26 
October  1973.  Further  reftnement  of  the  concept  occurred  in  1974. 

In  1975,  DoD  Directive  5000.28  was  issued  imposing  the  concept  of  DTC  on  all 
acquisitions  of  major  systems,  requiring  that  cost  be  weighted  equally  with  performance 
and  schedule.  According  to  the  directive,  DTC  has  a  twofold  objective: 

•  To  establish  cost  as  a  design  parameter  equal  in  importance  to  technical 
requirements  and  schedules  throughout  the  design,  development,  production, 
and  operation  of  the  system. 
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•  To  establish  cost  elements  os  management  goals  for  acquisition  managers  and 
contractors  to  achieve  the  best  balance  among  LCC,  acceptable  performance, 
and  schedule  [48]. 

In  recent  years,  OTC  has  been  used  less  frequently.  While  there  is  still  a  place  in  the 
SAR  for  the  design-to-cost  goal,  it  is  often  filled  by  an  “N/A”.  Of  the  64  programs  with 
FSD  start  in  the  1980s,  only  27  are  showing  a  DTC  goal. 

B.  BENEFITS  AND  WEAKNESSES 

The  primary  benefit  of  DTC  is  the  requirement  to  estimate  costs  throughout  the 
system’s  life  cycle.  Additional  benefits  are: 

•  DTC  defines  a  measurable  design  parameter  to  be  evaluated  along  with 
performance.  A  DTC  parameter  may  be  a  goal  or  a  threshold;  values  can  be 
expressed  in  constant  dollars,  resources  required,  or  other  measurable  factors 
that  influence  cost. 

•  DTC  provides  a  basis  for  communication  and  coordination  of  effort  between 
government  and  industry  paiticipants.  The  cost  goals  can  serve  as  a  “contact” 
between  the  program  manager  and  the  OSD  for  major  programs  or  tlie  services 
for  smaller  projects. 

•  DTC  leads  designers  and  production  engineers  to  take  a  design/production 
team  approach  during  the  design  process.  For  example,  the  A- 10  effort  by 
Fairchild  incorporated  the  design/production  team  approach  and  produced  its 
prototype  in  a  configuration  very  close  to  the  production  model. 

•  DTC  may  provide  easier  maintainability  through  simplicity  of  design.  Having 
to  meet  definitive  cost  goals  may  motivate  the  designer  to  look  for  the  simpler, 
more  maintainable  design. 

•  DTC  identifies  specifications  in  minimum  terms  of  performance,  thereby 
providing  the  contractor  with  leverage  to  make  cost-effective  tradeoffs. 

•  DTC  provides  strong  motivation  to  restrain  cost  growth. 

•  DTC  can  provide  an  early  idea  as  to  whether  or  not  cost  objectives  will  be  met. 

In  spite  of  all  the  expected  benefits,  the  DTC  concept  also  has  seme  weaknesses. 
These  are  explained  below  [49]: 

•  DTC  may  result  in  cost  goals  being  established  too  early.  DTC  forces  the 
program  manager  to  commit  to  a  DTC  goal  well  before  final  agreement  on 
configuration  and  operational  requirements.  Hence,  the  need  to  “sell”  the 
program  may  drive  DTC  goals  down  to  unrealistic  levels.  The  key  to  the 
success  of  the  DTC  concept  is  the  early  determination  of  a  specific  cost  goal; 
however,  it  may  be  extremely  difficult  to  maintain  a  goal  established  so  early  in 
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development.  Tradeoffs  are  made.  Test  results  may  change  the  dirertion  of  the 
development.  Reassessment  of  the  threat  may  alter  program  direction. 
Environmental  restrictions  could  alter  the  development  of  the  system.  Planned 
production  rates  may  change  in  response  to  the  results  of  iniiial  tests.  All  of 
these  items  could  drastically  affect  a  goal  based  on  a  paper  as.sessment.  So  one 
of  the  cornerstones  of  DTC  itself  represents  a  significant  weakness  of  the 
concept. 

•  DTC  may  stifle  innovation  and  restrict  the  use  of  new  technology.  A  contractor 
with  a  specified  cost  goal  tends  to  use  what  works,  rather  than  trying  a  new 
approach  that  may  reduce  costs  but  involves  risk. 

•  DTC  could  cause  suboptimization.  The  short-tem  goal  of  meeting  a  specific 
cost  ceiling  may  cause  decisions  tliat  ignore  long-term  cost  effects.  When 
budget  dollars  and  schedules  are  constrained,  it  is  easy  to  ignore  potential 
deficiencies  because  they  will  not  be  a  problem  for  several  budget  periods,  and 
then  they  will  be  someone  else’s  pioblem. 

•  DTC  results  in  performance  buy-in.  The  contractor  might  promise  superior 
performance  at  the  DTC  goal,  but  then  fail  to  match  claims  with  results  after 
getting  the  contraci.  This  moblem  can  be  partially  eliminated  through  the  use  of 
contractor  ‘  tlyofls”  or  prototypes  to  determine  how  well  promises  match 
results. 

•  DTC  imposes  cost  goals  that  arc  too  detailed.  If  goals  are  established  at  levels 
too  speed  c,  tile  benefits  of  DTC  in  contractor  ilexibility  and  cost  control  might 
be  adversely  affected.  The  more  that  is  specified,  the  less  flexibility  the 
contractor  has  in  meeting  cosi  objectives. 

•  DTC  may  increase  development  costs.  The  concept  requires  sufficient 
development  lime  and  money  to  be  used  successfully. 

•  Di  e  requires  additional  people,  lime,  and  effort  to  plan  and  execute  the 
pro  mam. 

DoD  Direciiv  -^45.3  requia-s  the  DTC  goal  to  be  established  before  Milestone  I  or  at  the 
earliest  praetie  .  date  thereaii.s-,  hut  in  no  case  should  the  goal  be  established  later  than 
entry  into  EMD  In  g-.-nera  wk  DT'"  concept  has  not  been  properly  applied.  It  has  not  been 
imoJemcntec  early  e  loug  t  in  c  concept  formulation  phase.  In  most  programs,  the  DTC 
goal  wa.s  not  iollowe  1  thi  »agh  i  completion.  It  either  was  dropped  or  faded  away  as  the 
program  progr  .ssed  Bradley  F  S.  Ml  lank,  F/A-18).  The  staff  of  the  Directorate  for 
Prot  urement  Policy  examined  ovt '  35  contracts  that  used  the  DTC  concept  and  found  that 
about  40  f  jCk  ent  haa  the  DTC  requirements  implemented  after  the  contract  was  executed 
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{50J.  For  example,  the  DTC  goal  for  the  F/A-18  was  implemented  after  the  program 
entered  the  EMD  phase. 

C.  ANALYSIS 

We  examined  and  compared  development  cost,  schedule,  production  cost,  and  total 
program  cost  growth  for  DTC  and  non  DTC  programs  in  three  areas:  across  programs,  by 
time  period,  and  by  service.  To  support  our  analysis  we  did  case  studies  on  three  aircraft 
programs,  the  F/A-18,  AH-64,  and  A- 10,  and  looked  at  three  additional  programs,  the 
MLRS,  Bradley  FVS,  and  the  Ml  tank.  More  information  on  the  case  studies  are  in  the 
previous  IDA  report  [1]. 

The  only  data  available  to  IDA  for  examination  of  cost  savings  due  to  use  of  a  DTC 
acquisition  siiategy  were  the  program  SARs  and  the  IDA  database  (developed  for  macro¬ 
analysis).  An  important  limitation  of  the  database  is  the  lack  of  information  on  operating 
and  support,  costs.  An  important  objective  of  DTC  is  to  reduce  life-cycle  costs.  We  did  not 
measure  the  impact  of  DTC  an  operating  and  support  costs.  Another  potential  impact  of 
DTC  that  we  could  be  missing  is  that  of  lower  co.sts  due  to  a  lower  cost  goal. 

A  properly-implemented  DTC  program  would  begin  during  demonstration/ 
validation,  before  the  development  estimate  »s  made.  Even  if  cost  growth  is  the  same,  DTC 
may  hold  costs  down. 

Table  VIII- 1  shows  the  outcomes  for  DTC.  Figure  VIII-1  shows  a  comparison  of 
cost  growth  for  programs  with  and  without  DTC  from  our  database. 


Table  Vllt-1.  Outcomes  for  Programs 
With  and  Without  DTC  (Percent) 


Outcome 

Measure 

With 

DTC 

Without 

DTC 

DCG 

36(39) 

PCG 

76(34) 

TPCG 

64  (34) 

DSG 

35  (39) 

DQG 

6(38) 

13  (58) 

Notes:  Numbers  in  parentheses  are  numbers  of 
programs.  Cost  growth  figures  are  weighted  by 
the  current  estimate. 


As  shown  in  Table  VIII-1,  total  program  cost  growth  in  DTC  programs  is  27 
percentage  points  greater  than  that  of  the  non-DTC  programs.  Both  development  and 
production  cost  growth  were  greater  in  DTC  programs,  while  there  were  no  major 
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differences  in  development  schedule  growth  and  development  quantity  growth.  The  overall 
statistics  of  the  sample  in  our  study  do  not  exhibit  cost  savings  being  associated  with  the 
implementation  of  the  DTC  acquisition  strategy.  This  indicates  that  DTC  has  not  been 
effective  as  practiced. 


Development  Production  Total  Program 

Cost  Growth  Cost  Growth  Cost  Growth 


■  With  DTC  P  Without  DTC 


Figure  VIII-1.  Average  Cost  Grovirth  for  Programs  With  and  Without  DTC 


However,  the  analysis  of  cost  and  schedule  outcomes  of  programs  with  and 
without  DTC  by  lime  period  (Figure  VIII-2)  indicates  that  DTC  programs  starting  in  the  late 
1970s  have  slightly  less  cost  growth  than  non-DTC  programs.  In  this  time  period  the  cost 
growth  of  the  DTC  programs  is  only  51  percent  and  that  of  the  non-DTC  programs  is 
55  percent.  This  may  indicate  that,  by  the  late  1970s,  the  DTC  concept  had  enough  time  to 
become  established  and  to  be  applied  early  enough  in  a  program  to  be  effective.  Total 
program  cost  growth  by  FSD  start  year  is  presented  in  Figure  VIII-3. 

We  also  examined  the  outcomes  of  programs  with  and  without  the  DTC  acquisition 
strategy  by  service — Army,  Navy,  and  Air  Force.  Our  analysis  shows  that  the  Army’s 
DTC  programs  have  less  cost  growth  than  those  without.  On  average,  the  Army’s 
production  and  total  program  cost  growth  are  higher  than  the  Navy’s  and  the  Air  Force’s 
overall.  For  the  Navy,  programs  with  DTC  have  higher  cost  growth  than  the  ones  without. 
The  Air  Force  program  outcomes  between  DTC  and  non-DTC  programs  are  about  the 
same.  The  results  of  this  analysis  are  shown  in  Figure  Vin-4. 
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(1)  (21)  (8)  (22)  p)  0)  (7) 
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Figure  VIII-2.  Total  Program  Cost  Growth  for  Programs 
With  and  Without  DTC  By  Time  Period 

To  better  evaluate  the  effectiveness  of  DTC  acquisition  strategy,  we  did  short  case 
studies  for  selected  programs  that  used  DTC  to  support  the  results  of  our  macro-analysis. 
DTC  programs  produce  mixed  results  in  the  program  outcomes.  Successful  cases  are  the 
MLRS  and  A- 10.  Apparently  the  methods  of  implementing  DTC  make  a  significant 
difference  in  its  success.  Inappropriate  implementation  has  substantially  reduced  the 
potential  effectiveness  of  DTC  [IJ. 

Among  our  findings  were: 

•  Most  of  the  early  systems  we  had  information  on  had  the  DTC  requirements 
forced  upon  them  as  a  retrofit,  after  initial  research  and  development  (R&D) 
contracts  were  awarded.  Because  of  this  retrofitting,  it  is  difficult  to  evaluate 
the  effectiveness  of  DTC. 

•  System  performance  is  still  the  first  priority.  Traditional  emphasis  on 
performance  and  schedule  resulted  in  a  relatively  low  priority  being  given  to 
cost  (e.g.,  F/A-18), 
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Figure  Vlli-3.  Total  Program  Cost  Growth  for  Programs 
With  and  Without  DTC  by  FSD  Start  Year 


160  T  155% 


(13)  (9)  (14)  (7)  (7)  (19) 

Army  Navy  Air  Force 

■  With  DTC  □  Without  DTC 


Note:  Numbers  in  parentheses  are  numbers  of  programs. 

Figure  VIII-4.  Total  Program  Cost  Growth  for  Programs 
With  and  Without  DTC  by  Service 
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•  DTC  has  been  used  mainly  as  a  cost-monitoring  device  in  FSD  rather  than  as  a 
tool  for  making  tradeoffs  earlier  in  the  process  (e.g.,  F/A-18). 

•  There  has  been  an  absence  of  continued  technical  evaluation  of 
design/effectiveness/cost  tradeoffs  throughout  the  program  acquisition  phase 
(e.g.,  Bradley  FVS). 

•  There  has  been  no  standardized  method  to  implement  DTC.  Each  DTC 
program  uses  its  own  management  approach  and  definition.  For  example: 

-  The  A- 10  program  introduced  a  life-cycle-cost  requirement  of  10  years,  but 
the  emphasis  was  on  meeting  the  stringent  unit  production  cost  goal  in  order 
to  ensure  program  support. 

-  The  Utility  Tactical  Transport  Aircraft  System  (UTTAS)  program  placed 
contractual  DTC  goals  and  incentives  on  average  airframe  production  cost. 

-  The  contractor’s  cost  model  for  the  CH-47  modification  program  did  not 
include  the  impact  of  tradeoffs  in  achieving  DTC  unit  production  goals  on 
operation  and  support  costs. 

-  The  F/A-18  program’s  DTC  value  was  based  on  a  cumulative  average 
recurring  cost  for  8(X)  aircraft. 

-  The  AH-64A  program’s  DTC  cost  goal  was  based  on  the  production  cost 
for  the  A- 10  airframe. 

Generally,  DTC  targets  (affordability  limits)  were  not  established  during  concept 
formulation,  when  the  greatest  flexibility  existed  to  tradeoff  performance  for  the  dollars 
available  (AMST,  UTTAS,  and  CH-47  modification)  [52]. 

DTC  has  the  potential  to  produce  significant  cost  reductions  if  problems  of 
implementation  could  be  resolved.  The  MLRS  is  a  good  example  of  a  successful 
implementation  of  the  DTC  philosophy.  TTie  main  factor  that  contributed  to  the  success  of 
the  MLRS  is  the  establishment  of  a  realistic  but  still  challenging  goal.  The  key  success  of 
the  implementation  of  the  MLRS  DTC  goals  was  based  on  the  following  principles:  stable 
specification  and  requirement  documents,  establishment  of  a  design-to-unit-production-cost 
(DTUPC)  goal  for  the  total  program  and  by  fiscal  year,  compatibility  of  DTUPC  quantities 
with  procurement  implementation  plans,  and  costs  and  designs  based  on  existing 
production  technologies  [53]. 
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D.  FINDINGS  AND  RECOMMENDATIONS 


Based  on  our  analyses  of  DTC,  particularly  the  analysis  of  specific  cases,  we 
recommend  the  following: 

•  Establish  DTC  goal  early.  The  goal  must  be  established  before  the  start  of  the 
validation  phase,  because  it  provides  a  baseline  to  work  against  in  the  tradeoff 
decisions,  which  occur  during  validation. 

•  Be  flexible  about  the  design.  The  number  of  specified  performance  parameters 
should  be  minimized  in  DTC.  They  should  also  be  ranked  according  to 
priority,  if  possible. 

•  Use  new  technology  to  lower  cost  rather  than  to  increase  performance.  This 
requires  a  change  on  the  part  of  engineers  who  for  years  have  been  encouraged 
to  rank  performance  over  cost. 

•  Allocate  the  DTC  goal  down  the  work  breakdown  structure  and  track  costs 
regularly  for  both  prime  contractor  and  subcontractor  efforts.  The  DTC  goal 
should  be  related  to  quantity  from  Unit  1  on  up;  setting  a  DTC  goal  for  Unit  1 
imposes  strict  discipline  on  the  designer  and  permits  an  early  indication  of 
compliance. 

•  Use  contractual  incentives.  Contractual  innovations  are  needed  to  give  the 
contractor  an  incentive  to  build  a  reliable,  low-cost  product.  Reliability 
Improvement  Warranties  and  award  fees  are  two  such  devices. 

•  Allocate  enough  time  and  money  to  implement  DTC.  DTC  should  require  that 
adequate  time  and  sufficient  funds  are  available  during  development  to  permit 
examination  of  tradeoffs  and  alternative  design  approaches.  Constrain  5  either 
may  cause  suboptimization. 

•  Establish  realistic  cost  objectives.  The  goal  should  reflect  the  best  estimate 
based  on  available  data. 

Most  DTC  programs  in  the  sample  applied  DTC  either  as  a  retrofit  or  too  late  in  the 
development  phase  (full-scale  development)  to  be  cost-effective.  Other  acquisition 
programs  implemented  DTC  at  the  beginning,  and  as  the  programs  progressed,  dropped  the 
strategy  due  to  additional  advanced  technologies  applied  to  the  system’s  design  because  of 
requirements  changes  (e.g.,  Bradley.  Ml  tank).  Our  macro-analysis  indicates  that  cost 
growth  is  greater  for  programs  with  DTC  than  for  programs  without,  except  in  the  late 
197()s.  This  exception  may  be  because  the  DTC  concept  had  become  well  enough 
established  by  the  late  1970s  to  be  implemented  earlier  in  the  programs. 
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IX.  DUAL-SOURCING 


A.  BACKGROUND 

Defense  acquisition  has  a  long  history  of  competition.  The  Armed  Services 
Procurement  Act  of  1947  required  that  contracts  for  property  or  services  be  formally 
advertised.  0MB  Circular  A- 109  directs  that  competition  be  used  throughout  a  program, 
particularly  during  design  and  development.  Competition  in  design  and  development  has 
the  advantage  of  allowing  the  exploration  of  different  alternatives.  Competition  often  has 
been  used  in  full-scale  development.  More  recently,  however,  the  government  has 
emphasized  dual-sourcing  in  production,  the  explicit  goal  being  lower  prices  and,  possibly, 
better  performance. 

In  the  1980s,  Congress  prescribed  production  competition.  Iw  the  Defense 
Appropriations  Act  of  1984,  Congress  required  that  any  ma  jor  acquisition  program  have 
either  a  certification  that  the  system  would  be  procured  in  insufficient  quantities  to  wan'ant 
multiple  sourcing  or  a  plan  for  the  development  of  two  or  more  sources.  The  Competition 
in  Contracting  Act  (CICA)  of  1984  established  requirements  for  maximizing  competition. 
Competition  was  to  be  the  norm;  exceptions  were  to  be  justified.  CICA  required  the 
appointment  of  competition  advocates  to  review  acquisition  strategies.  It  both  provided  for 
specific  procedures  designed  to  guarantee  that  all  sellers  could  bid  for  a  proposed 
procurement  and  established  protest  procedures.  Additional  legislation — the  Department  of 
Defense  Procurement  Reform  Act  and  the  Small  Business  and  Federal  Procurement 
Competition  Enhancement  Act  of  1984  and  the  Defense  Procurement  Improvement  Act  of 
1985 — also  aimed  to  increase  competition  in  defense  contracting. 

In  addition,  the  Defense  Department  has  encouraged  competition.  The  Defense 
Acquisition  Improvement  Program  (the  Carlucci  initiatives),  instituted  in  1981,  includes  an 
initiative  to  increase  competition  in  the  acquisition  process.  The  Packard  Commission 
recommended  the  use  of  commercial-style  competition.  It  recommended  development  of  a 
waiver  before  hardware  could  be  uniquely  developed  for  the  military.  In  1984,  the  Defense 
Systems  Management  College  published  a  handbook  for  program  managers  on  enhancing 
competition  [54], 
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In  1989,  the  Naval  Center  for  Cost  Analysis  examined  twelve  dual-sourced 
acquisition  programs  to  determine  the  extent  of  any  savings  [55].  All  twelve  programs  had 
net  savings,  ranging  from  0.8  percent  to  27.9  percent,  with  an  average  net  savings  across 
the  twelve  programs  of  14.2  percent.  Savings  of  less  than  5  percent  were  estimated  for  the 
AIM-54C  Phoenix  missile  and  the  F404  aircraft  engine.  For  both  of  those  programs,  dual¬ 
sourcing  was  started  late  in  the  production  span,  and  because  of  the  technical  complexity  of 
both  programs,  there  were  very  few  firms  that  could  qualify  as  second  sources. 
Contractors  on  two  other  programs  experienced  severe  financial  problems,  with  mixed 
results  on  the  program  outcomes.  Savings  on  the  TAO-187  program  were  estimated  to  be  a 
below-average  7.5  percent  because  of  the  bankruptcy  and  shutdown  of  Pennsylvania 
Shipbuilding,  which  led  to  additional  costs  for  refurbishment  and  construction  of  partially 
completed  vessels.  In  contrast,  the  LSD-41  program  had  the  highest  estimated  net  savings 
(27.9  percent)  in  spite  of  financial  difficulties  of  the  lead  builder,  Lockheed  Shipbuilding 
and  Construction;  the  last  five  ships  were  built  by  Avondale,  with  large  savings  after  a 
winner-lake-all  competition.  IDA  has  also  reviewed  additional  studies  on  dual-source 
programs  and  recommended  criteria  for  analysis  [56]. 

The  threat  changes  of  the  1990s  and  the  resulting  reductions  in  funding  available  to 
the  DoD  imply  major  changes  in  the  amount  of  dual-sourcing  that  is  likely  to  occur.  For 
many  programs,  there  will  be  barely  enough  funding  to  sustain  a  single  source.  For  new 
programs,  there  might  be  competitive  prototyping,  but  sole-source  production.  In  any 
event,  stricter  criteria  for  dual-source  programs  are  likely. 

In  this  study,  we  focused  on  competition  as  dual-sourcing  in  production  for  major 
weapon  systems  and  subsystems.  This  type  of  competition  typically  requires  that  the 
government  have  a  hand  in  developing  an  alternative  source,  just  as  it  developed  the  first 
source.  Other  methods  of  enhancing  competition  in  major  weapons  systems,  including 
winner-lake-all  or  vendor  competition,  are  not  discussed  here. 

B.  BENEFITS  AND  WEAKNESSES 

We  can  think  of  the  types  of  items  that  the  government  buys  as  being  along  a 
continuum  with  re.spcct  to  quantity  and  complexity.  Small,  uncomplicated  items  that  the 
government  buys  a  lot  of  over  the  years  are  easy  to  compete.  In  many  cases.  Uiese  items  ate 
standardized,  and  obtaining  multiple  sources  is  relatively  easy.  At  the  other  end  of  the 
continuum,  major  weapon  systems  are  developed  on  a  customized  basis  and  produced  in 
relatively  small  numbes.  A  company  that  wants  to  produce  Sidewinder  missiles  cannot 
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merely  do  some  quick  tooling  and  start  producing  them — a  detailed  technical  data  package 
is  needed. 

Government  funding  of  the  second  source  is  fairly  common,  as  is  the  use  of 
‘educational  buys”  or  “qualification  buys”  to  get  the  second  source  started.  Another 
common  factor  is  the  use  of  annual  competitive  bids  between  contractors.  Typically,  in 
major  systems,  these  annual  competitions  are  not  winner- take-all,  but  the  buy  is  split 
between  the  contractors.  This  practice  requires  the  government  to  fine-tune  its  approach:  the 
government  must  give  the  winner  enough  of  a  “reward”  to  encourage  future  low  bids,  but  it 
must  give  the  loser  a  large  enough  order  to  keep  its  production  line  going.  Depending  on 
how  its  bid  is  structured,  a  clever  loser  might  end  up  with  more  profits  than  a  winner. 

In  the  short  term,  the  expected  benefits  of  dual-source  competition  in  majoi  systems 

include: 

•  Lower  overall  costs 

•  Increased  contractor  responsiveness  to  government  needs 

•  Enhanced  system  quality  and  reliability,  put  in  as  an  attractive  feature  for 
government  purchasers.  (A  number  of  competitive  programs  we  studied, 
including  Tomahawk  and  the  alternative  fighter  engine,  were  motivated  more 
by  quality  considerations  than  by  cost.) 

Longer-term  benefits  could  include: 

•  Enhanced  industrial  base  for  particular  systems 

•  Increased  capital  investment  by  contractors. 

In  the  short  term,  the  weaknesses  of  competition  include  additional  costs  in  areas 
not  found  in  single-source  production: 

•  Dual-sourcing  typically  requires  an  up-front  investment  for  tooling,  equipment, 
qualification,  and  administration  to  establish  a  second  source. 

•  By  splitting  a  buy  between  two  contractors,  the  government  may  give  up  some 
economies  of  scale  because  the  full  benefits  of  learning  and  high-rate 
production  are  not  realized.  Large  buys  typically  exhibit  lower  unit  costs  than 
small  buys. 

•  If  multiple  configurations  are  required,  support  costs  may  increase. 

Little  attention  has  been  paid  to  the  possibility  of  long-term  weaknesses  of 
competition  with  re.spect  to  the  relationships  between  industry  and  the  Department  of 
Defense.  Are  the  benefits  of  dual-sourcing  a  one-time  effect,  or  can  they  be  sustained  over 
lime?  Production  competition  in  major  systems  must  be  viewed  as  an  investment  decision. 
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The  potential  reduction  in  procurement  costs  must  be  weighed  against  additional  up-front 
costs  and  increased  government  administrative  costs.  This  tradeoff  is  unique  for  each 
program. 

The  Defense  Systems  Management  College  program  managers*  handbook  indicates 
a  method  for  evaluating  the  impact  of  production-level  competition  on  program  costs.  In 
evaluating  potential  or  actual  cases  of  competition,  policymakers  may  find  these  guidelines 
useful  [54.  pp.  7-1,  7-2]: 

(1)  Estimate  single  source  recurring  production  costs  by  fiscal  year  in  constant  dollars 
based  upon  progress  curves  and  expressed  as  conuactor  price. 

(2)  Estimate  competitive  recurring  production  costs  by  fiscal  year  in  constant  dollars 
based  upon  progress  curves.  Reasonable  assumptions  must  be  made  concerning  shift 
and  rotation  and  the  second  source  progress  curve. 

(3)  Calculate  potential  savings  by  subtracting  (2)  from  (1)  by  fiscal  year. 

(4)  Calculate  net  potential  savings  by  subtracting  annual  incremental  government  costs, 
staled  in  constant  dollars,  from  (3). 

(5)  Estimate  nonrecurring  start-up  costs,  stated  in  constant  dollars,  by  fiscal  year. 

(6)  Estimate  incremental  logistic  support  costs,  stated  in  constant  dollars,  by  fiscal  year. 

(7)  Calculate  a  net  present  value  of  competitive  versus  sole  source  production  costs  by 
subtracting  the  discounted  costs  (5)  and  (6)  from  the  di.scounted  benefits  (3). 

(8)  Compare  discounted,  constant,  and  then-year  dollar  estimates  of  single  source  and 
comitelitive  pnxluction. 

(9)  Conduct  detailed  sensitivity  analyses  to  investigate  the  effect  of  changes  in  key 
assumptions  on  the  estimate  of  savings,  and  to  develop  a  range  of  likely  estimates. 

C.  ANALYSIS 

Depending  on  the  appropriateness  and  availability  of  data,  we  conducted  an  analysis 
of  the  program  database  described  in  Section  III,  including  statistical  analyses  of  cost 
growth  estimates.  Table  IX- 1  shows  the  outcomes  for  the  dual-sourced  programs. 

We  see  considerable  variability  in  the  outcomes  for  dual-sourced  programs.  One 
unifying  factor  is  low  cost  growth  in  ship  programs  (except  the  FFG-7).  During  the  time 
that  these  ships  were  acquired,  dual-sourcing  in  ship  programs  was  virtually  universal.  It 
was  also  a  time  of  considerable  overcapacity  in  the  shipbuilding  industry.  Dual  sourcing 
interacted  with  industry  conditions  to  create  favorable  outcomes. 
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Tabis  IX-1.  Outcomas  for  Dual>Sourcod  Program*  (Parcant) 


Procram 

PCG 

TPCG 

PSG 

Stretch 

Heliriie 

71 

44 

75 

131 

-24 

Sparrow-F  (AIM-7F) 

58 

75 

22 

66 

-26 

TDW 

78 

70 

127 

■41 

285 

Sidewincier-L  (AIM-9L) 

107 

125 

176 

23 

125 

UR  Maverick 

55 

51 

36 

-37 

114 

Spairow-M  (AIM-7M) 

28 

26 

61 

41 

14 

Sidewinder-M  (AIM-9M) 

1 

10 

144 

127 

7 

Phoenix-C  (A1M-54C) 

98 

92 

.50 

252 

-57 

Shillelagh 

54 

47 

44 

-11 

61 

MK  48  ADCAP 

0 

2 

53 

0 

53 

Stinger  Basic/POST 

84 

87 

-18 

■<50 

104 

Dragon 

172 

159 

13 

-73 

313 

Standard  Missile  2 

-9 

-4 

67 

36 

22 

Tomahawk 

57 

45 

29 

235 

-61 

SURTASS/T-AGOS 

63 

68 

.54 

.50 

3 

CG-47 

-6 

-6 

14 

69 

-33 

DDG-51 

-6 

-1 

111 

322 

-.50 

FFC.-7 

40 

40 

83 

2 

80 

LHD-1 

-6 

49 

100 

-26 

LSD.41 

-8 

-8 

0 

-33 

.50 

SSN-688 

-1 

-1 

60 

520 

-74 

T-AO-187 

-8 

-8 

2 

6 

-3 

Table  IX-2  shows  the  results  from  the  aggregate  analysis.  An  analysis  of  averages 
from  the  full  sample  and  from  the  group  of  tactical  munitions  and  ships  (where  virtually  all 
the  dual-sourcing  in  our  group  of  programs  has  occurred)  is  interesting.  In  production, 
where  one  would  expect  dual-sourcing  to  have  the  greatest  impact,  production  outcomes 
were  considerably  more  favorable  for  dual-sourced  tactical  munitions.  Production  cost 
growth  for  dual-sourced  tactical  munitions  was  only  54  percent,  while  it  was  159  percent 
for  non-dual-sourced  tactical  munitions. 

We  also  see  that  production  quantity  growth  is  much  higher  in  the  dual-sourced 
programs,  and  program  stretch  is  much  lower.  It  appears  that  stable  programs  are  chosen 
for  dual-sourcing,  or  (more  likely)  that  the  government  keeps  competitive  programs  more 
stable,  in  order  to  protect  its  up-front  investment. 

Another  equipment  type  with  considerable  dual-sourcing  is  ships.  Production  cost 
growth  for  dual -sourced  ships  was  4  percent  on  average,  while  it  was  29  percent  for  the 
two  non-dual-sourced  ship  programs.  This  adds  *o  our  favorable  results  for  dual-sourcing. 
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Table  iX-2.  Outcomes  for  Programs  With  and  Without  Dual-Sourcing  (Percent) 

_ Ships _ Tactical  Munitions _ All  Programs 
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However,  it  is  important  not  to  make  too  much  of  the  ship  results,  because  the  two  non¬ 
dual-sourced  programs  were  considerably  older.  During  the  1970s  and  1980s,  ship  dual- 
sourcing  was  the  rule,  and  the  shipbuilding  industry  had  considerable  overcapacity.  Dual¬ 
sourcing  probably  contributed  to  the  strongly  favorable  outcomes  for  ships  relative  to  other 
equipment  types. 

D,  FINDINGS  AND  RECOMMENDATIONS 

The  findings  of  our  analyses  of  dual-sourced  programs  are  summarized  below; 

•  In  missile  programs,  cost  growth  in  dual-sourced  programs  was  lower  than  in 
non-dual-sourced  programs.  While  this  difference  was  not  statistically 
significant,  it  suggests  that  dual-sourcing  may  be  beneficial.  In  ship  programs, 
production  cost  outcomes  were  also  favorable,  but  ship  dual-sourcing  was  the 
rule.  We  do  not  have  a  good  control  group  for  the  analysis,  but  dual-sourcing 
may  have  contributed  to  the  strongly  favorable  outcomes  for  ships. 

•  It  is  easier  to  find  savings  in  prices  than  in  resource  inputs.  .Several  studies 
evaluating  costs  such  as  engineering  hours  and  manufacturing  inputs  find 
similar  direct  resource  usage  in  dual-source  and  sole-source  programs.  Thus, 
savings  from  dual-sourcing  seem  to  come  from  eitliei  profits  or  the  prices  for 
inputs. 

•  Dual  sourced  programs  tend  to  buy  more  quantity  than  planned  over  a  longer 
period  of  time  than  planned.  This  tends  to  amortize  development  costs  and 
second-source  startup  costs.  It  may  be  that  the  benefits  seen  from  dual¬ 
sourcing  are  really  benefits  of  program  stability — this  is  a  chicken-egg 
problem. 

•  Cross-program  effects  and  industry  strategies  have  been  insufficiently 
analyzed.  Even  if  we  see  savings  from  dual-sourcing,  we  also  need  to  examine 
whether  there  are  cost  increases  in  sole-source  programs  produced  in  the  same 
plant.  Also,  in  .some  programs,  such  as  Hellfire,  we  see  a  seesaw  pattern  of 
production,  with  the  companies  alternating  winning  the  major  share  of  the 
year’s  production.  This  pattern,  if  it  is  regular,  allows  contractors  to  plan  stable 
production  rates.  One-shot  gains  may  be  possible  as  dual-sourcing  represents  a 
shock  to  the  system — it  is  unclear  whether  or  not  such  gains  could  be  sustained 
were  dual-sourcing  to  become  a  universal  acquisition  strategy. 

(These  findings  pertain  to  major  systems.  For  subsystems,  the  dual-  or  multiple¬ 
sourcing  picture  is  quite  different.  Up-front  investments  are  typically  smaller  than  for  full 
systems,  the  number  of  items  being  procured  is  often  larger,  and  the  items  are  frequently 
less  complicated.) 
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Based  on  our  findings,  we  cun  make  the  following  recommendations  regarding 
dual-source  competition  in  major  systems: 

•  Dual-source  production  should  not  be  prescribed  across  the  board  for  major 
systems.  Competition  can  be  of  value  in  particular  individual  cases;  however,  it 
is  very  difficult  to  predict  what  those  cases  are.  Based  on  our  analysis,  we  can 
make  the  observation  that  conditions  of  industry  overcapacity  appear  to  be 
favorable  to  competition.  The  larger  and  more  customized  the  system,  and  the 
lower  the  quantity,  the  harder  it  is  for  dual-sourcing  to  be  viable  because  of  the 
larger  investment.  Additional  work  needs  to  be  done  on  criteria  for 
competition. 

•  That  cost  savings  from  comnetition  are  uncertain  should  be  recognized.  It  does 
not  make  sense  to  plan  on  large,  immediate  cost  savings.  Dual-sourcing 
requires  some  up-front  investment,  and  payback  occurs  over  a  number  of 
years. 

•  Specific  guideline;,  should  be  established  for  dual-sourcing,  similar  to  tho.se  for 
MYP.  Competition  is  best  applied  under  the  following  conditions: 

a  large  number  of  systems  are  required, 

a  firm  plan  and  stable  funding  are  available, 

break-even  analysis  suggests  that  costs  can  be  recovered  over  a  reasonable 
period, 

technology  transfer  involved  is  relatively  straightforward,  and 
adverse  effects  on  other  programs  arc  negligible. 

•  Benefits  other  than  reduced  prices  may  exist  and  need  to  be  considered. 
Among  these  are  increased  contrector  responsiveness,  increased  system 
reliability,  and  preservation  of  the  industrial  base. 

•  Additional  research  should  be  done  into  the  long-term  effects  of  dual-sourcing. 
Such  re.search  should  go  beyond  the  individual  program  to  consider  overall 
contractor  su  ategies  that  can  alfecl  the  cost  of  other  programs. 


X.  TOTAL  PACKAGE  PROCUREMENT 
AND  FIXED-PRICE  DEVELOPMENT 


We  analyzed  lotal-package  procurement  and  fixed-price  development  together 
because  they  have  common  implications — both  shift  risk  from  the  government  to  the 
contractor.  Total  package  procurement  involves  considerable  risk  for  the  contractor  in  both 
development  and  production,  while  fixed-price  development  involves  risk  in  development, 
but  not  in  production.  Total-package  procurement  was  begun  in  the  1960s,  and  fixed-price 
development  was  begun  in  the  i980s. 

A  .  TOTAL  PACKAGE  PROCUREMENT 

Total  package  procurement  (TPP)  was  one  of  the  first  major  initiatives  developed 
by  the  DoD  as  an  effort  to  restrain  cost  overruns  in  the  weapon  system  acquisition  process. 

1 .  Background 

In  the  mid-1960s,  successful  development  contracts  were  generally  followed  by 
production  contracts.  Little  or  no  likelihood  existed  that  the  develofjer  would  have  to  face 
competition.  Thus,  the  coniracior.*;  had  incentives  to  “buy  in” — to  underestimate  the  cost  of 
development  programs  in  order  to  win  the  development  contract  and  place  themselves  in  a 
sole-source  position  for  the  much  larger  follow-on  production  contracts. 

To  attack  thi.s  problem,  Robert  H.  Charles,  the  Assistant  Secretary  of  the  Air  Force 
at  the  time,  designed  the  total  package  prtxiuremeni  concept  The  objectives  of  this  concept 
were  to; 

•  Limit  or  eliminate  buy-in  considerations, 

•  Motivate  contractors  to  design  for  economical  production  and  enforce  design 
discipline, 

•  Encourage  subcontracts  with,  and  obtain  components  from,  the  mo.st  efficient 
.supply  sources. 
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•  Obtain  long-term  commitments  leading  to  program  stability  and  continuity,  and 

•  Encourage  contractor  efficiency  through  competition  and  thereby  reduce  costs. 

According  to  Charles,  TPP  would  allow  the  government,  like  any  buyer  in  the 
commercial  world,  “to  make  a  choice  between  competing  products  on  the  basis,  not  of 
estimates,  but  of  binding  commitments  concerning  performance  and  price  of  operational 
equipment”  [57], 

TPP  required  contractors  to  bid  on  the  development,  production,  and  spare-part 
support  work  under  one  contract.  Price  and  performance  commitments  were  obtained 
during  the  contract  definition  phase.  The  purpose  of  the  TPP  contract,  generally  of  a  fixed- 
price  incentive  type,  was  to  offer  the  government  the  opportunity  to  shift  the  major  risk  and 
major  program  management  responsibility  to  contractors. 

The  results  of  the  TPP  initiative  fell  far  short  of  the  goals.  Cost  overruns  continued, 
new  defense  systems  failed  to  meet  technical  performance  requirements,  and  schedules 
slipped  on  many  programs.  The  reasons  for  the  failure  of  the  TPP  concept  are  many.  The 
onset  of  inflationary  pressures  in  the  economy  during  the  Vietnam  era — unrelated  to  a 
specific  program — may  have  been  partially  responsible  for  the  failure  of  the  TPP  concept. 
More  importantly,  TPP  did  not  provide  contractors  with  sufficient  management  flexibility 
to  cope  with  problems  as  they  became  known.  Contractors  had  to  make  substantial 
production  commitments  to  meet  delivery  schedules  before  completion  of  design  and 
verification  by  testing.  Costly  redesign  and  rework  followed.  Continued  tradeoff  analysis 
was  stifled  because  of  the  rigidity  of  the  contracts.  Finally,  the  government  typically  did  not 
enforce  the  total  package  procurement  contract  provisions  if  the  contractor  failed  to  achieve 
the  program’s  goals.  When  problems  occurred,  the  total  package  procurement  contracts 
were  converted  to  cost-reimbursement  type  contracts,  and  the  contractors  were  required  to 
take  substantial  losses  on  the  program. 

Although  the  Air  Force’s  Maverick  air-to-surface  missile  program  successfully  used 
the  TPP  concept,  serious  problems  were  encountered  on  many  other  total  package 
procurement  programs.  Among  those  troubled  programs  were  the  Air  Force  Galaxy  C-5A 
transport  and  Short-Range  Attack  Missile  (SRAM);  the  Army  Cheyenne  AH-56A 
helicopter,  which  was  canceled;  and  the  Navy  DD-96.'^  destroyer  and  LHA-1  helicopter 
carrier.  As  a  result  of  the  problems  encountered,  DoD  recognized  the  need  to  place  .stringent 
limitations  on  the  application  of  TPP.  By  1972,  TPP  was  abolished  by  the  Deputy 
Secretary  of  Defense. 
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2.  Benefits  and  Weaknesses 


The  expected  benefits  of  the  TPP  concept  at  the  time  of  its  introduction  included: 

•  Better  definition  of  design  specifications.  The  TPP  concept  requires  design 
tightening  and  configuration  discipline  on  the  part  of  the  contractor.  At  the 
same  time,  it  forces  the  government  to  define,  early  in  the  program,  specifically 
what  it  wants. 

•  Less  unrealistic  “salesmanship”  or  “buy-in”  bidding.  TPP  was  designed  to 
allow  the  Department  of  Defense  to  make  a  choice  among  competing 
contractors  on  the  basis  of  binding  commitments  on  the  performance  and  price 
of  the  system. 

•  Better  contractor  commitment  to  design  for  economical  production,  reliability, 
and  maintainability. 

•  More  efficient  selection  of  vendors.  The  contractor  is  motivated  to  obtain 
supplies  from  the  most  efficient  sources. 

•  Less  need  for  subsequent  competitive  reprocurement  of  components.  This  is 
due  to  increased  competition  at  program  initiation. 

•  More  efficient  contractors.  The  winning  contractor  is  more  efficient  due  to 
tougher  competition  at  the  beginning  of  the  program. 

•  Better  long-range  planning  required  by  both  the  government  and  the  contractor. 

Despite  these  promises,  the  TPP  concept  has  some  weaknesses.  By  attempting  to 
fix  a  price  on  a  paper  concept  for  a  future  system,  TPP  fails  to  recognize  risks  involved  in 
taking  a  design  from  paper  to  reality — potential  for  cost  growth  and  technological  risks. 
The  costs  identified  by  the  contractor  are  only  estimates  and  should  be  treated  that  way — 
provisions  should  be  made  for  periodic  updating.  Also,  the  specific  definitions  of 
performance  requirements,  schedules,  and  production  quantities  restrict  the  contractor’s 
ability  to  perform  in  the  most  cost-effective  way  when  moving  from  design  to  actual 
Hardware  [58]. 

3.  Analysis 

Our  analysis  shows  that  the  total  cost  growth  in  TPP  programs  is  greater  than  the 
cost  growth  in  non-TPP  programs  by  28  percentage  points.  The  outcomes  of  TPP 
programs  and  the  outcomes  of  the  TPP  versus  non-TPP  programs  are  presented  in  Table 
X-1  and  Table  X-2,  respectively. 

Cost  growth  in  development  and  in  production  is  substantially  higher  in  total 
package  programs.  TPP  programs  also  take  longer  than  planned  to  reach  IOC. 
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TabI*  X-1.  Outcomes  for  TPP  Programs  (Percent) 


Systems 

PCG 

TPCG 

PSG 

_  _PCX5 

C-5A 

115 

77 

19 

-34 

SRAM 

456 

263 

— 

114 

Maverick  AGM-65A 

-1 

1 

14 

18 

SURTASS  T/AGOS 

63 

68 

54 

50 

LHA 

58 

57 

80 

-44 

DDG-963 

23 

23 

78 

3 

Table  X-2.  Outcomes  for  Programs 
With  and  Without  TPP  (Percent) 


Outcome 

Measure 

With  TPP 
(N=^ 

Without  TPP 
(N=76) 

DCG 

128 

27 

PCG 

91 

53 

TPCG 

72 

44 

DSC. 

.54 

34 

PSG 

49 

57 

PQG 

18 

35 

Note:  Cost  growth  figures  are  weighted  by  the  euirent 


estimate. 


While  the  goal  of  the  TPP  concept  was  desirable,  the  quantum  leap  in  acquisition 
practice  that  implementation  of  the  concept  represented  was  a  factor  in  its  failure.  TPP  was 
an  effort  to  change  30  years  of  acquisition  in  a  single  step.  It  depended  on  details  and 
projections  never  before  attempted  for  large  programs.  For  example: 

•  It  required  detailed  specification  of  performance  requirements,  schedules,  and 
production  quantities  before  a  single  piece  of  hardware  had  been  built. 

•  It  required  that  the  contractors  project  requirements  far  into  the  future  and 
provided  no  provision  for  revision. 

•  It  attempted  to  set  a  firm  price  on  the  development  and  production  of  a  complex 
system  before  any  part  of  that  system  had  been  constructed. 

The  total  package  procurement  concept  shifts  the  major  role  of  government 
personnel  from  the  acquisition  phase  to  the  conceptual  and  definition  phases.  TPP  shifts 
risk  from  the  government  to  the  contractor.  To  a  certain  extent,  risk  shifting  may  be  a  good 
idea.  In  private  industry,  companies  that  develop  products  bear  all  the  risk.  If  those 
products  are  not  acceptable  to  consumers,  the  firms  are  unable  to  sell  them,  and  they  lose 
money.  Ultimately,  these  firms  may  fail.  However,  in  major  weapons  systems,  the  product 
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is  developed  on  a  customized  basis  for  the  government.  If  the  government  shifts  too  much 
risk  onto  the  contractor  and  there  is  an  overrun,  the  contractor’s  existence  may  be 
jeopardized.  Then,  the  government  has  to  decide  whether  to  let  the  contractor  go  out  of 
business  or  bail  it  out.  In  the  cases  of  the  C-5A,  the  DD-963,  and  the  LHA-1,  the 
government  decided  to  biil  out  the  contractors.  Further,  if  contractors  know  that  they  are 
going  to  be  bailed  out,  then  major  risk-shifting  becomes  less  effective. 

4.  Findings  and  Recommendations 

The  total  package  procurement  concept  is  a  failure.  It  should  not  be  implemented 
except  in  rare  circumstances.  TPP  is  most  likely  to  have  serious  adverse  effects  on 
innovation  and  quality  in  systems  developments  where  the  requirement  is  uncertain,  the 
need  is  extremely  urgent,  die  iechnology  that  must  be  used  is  unproven,  or  the  measures  of 
systems  effectiveness  are  diffuse  and  qualitative. 

The  total  package  procurement  concept  might  have  been  successful  if  it  had  been 
implemented  in  a  more  orderly  fashion,  and  if  adequate  time  had  been  provided  for  both  the 
government  and  the  contractors  to  develop  an  understanding  of  the  implications  of  the 
concept — the  concept  was  introduced  in  mid- 1964  and  was  first  applied  to  a  major  system  a 
few  months  later. 

In  order  to  implement  TPP,  the  following  conditions  should  exist: 

•  The  system  should  be  thoroughly  and  clearly  defined  in  a  contract  definition 
phase. 

•  The  program  should  be  a  low-risk  development. 

•  The  project  should  be  short-term  (5  years  or  le.ss). 

•  An  announcement  .should  be  made  at  the  outset  of  the  program  that  substantial 
changes  are  not  permitted. 

The  Maverick  AGM-65A  proved  that  TPP  can  work  under  the  right  conditions. 

Because  most  of  the  SAR  programs  are  high-risk,  it  should  generally  not  be 
applied.  In  particular,  if  the  government  and  the  contractor  cannot  agree  on  a  stable  system 
definition,  the  TPP  initiative  should  not  be  applied. 

B.  FIXED-PRICE  DEVELOPMENT 

Fixed-price  development  (FPD)  was  instituted  by  the  Navy  in  order  to  encourage  an 
efficient  development  process  without  the  major  risk  of  total  package  procurement. 
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In  a  fiim-fixed-price  (FFP)  contract,  the  contractor  theoretically  accepts  all  risks  in 
exchange  for  the  stated  price.  The  government  is  required  to  make  no  price  adjustment  for 
the  original  work  after  the  contract  is  awarded,  regardless  of  the  contractor’s  actual  cost  in 
meeting  requirements.  Exceptions  occur  in  cases  of  government- approved  contract  changes 
made  in  response  to  changes  in  military  requirements,  technology,  a.id  funding.  Changes 
may  also  be  made  in  cases  involving  the  “Truth  in  Pricing”  lav,'.  This  law  makes  provisions 
in  cases  where  the  contractor  did  not  disclose  information  available  at  the  time  of  the 
negotiation,  causing  inaccurate  estimates.  Tlie  firm-fixed-price  conU'act  can  be  defined  as  a 
contract  that  specifies  a  certain  amount  to  be  paid  for  the  designated  system,  regardless  of 
the  contractor’s  cost  experience. 

1 .  Background 

The  practice  of  firm-fixed-price  contracting  by  the  DoD  has  gone  through  many 
changes  over  the  past  twenty  years.  In  1952  fixed-price  contracts  represented  82  percent  of 
defense  prime  contract  awards.  By  1961  this  had  dropped  to  58  percent.  Defense  Secretary 
Robert  McNamara  raised  the  percentage  to  79  percent  in  1966;  by  1970  the  fixed-price 
percentage  had  dropped  to  74  percent. 

Although  the  percentage  of  fixed-price  contracts  is  partly  a  function  of  the  type  of 
work  to  be  performed,  it  is  heavily  influenced  by  Pentagon  policy.  It  is  used  to  shift  the 
burden  of  cost  control  from  the  DoD  to  the  contractor.  After  1969,  tne  Laird-Packard 
influence  led  to  a  decline  in  fixed-price  contracting  and  an  increase  in  the  use  of  cost- 
reimbursement  contracts  for  research  and  development  work.  The  Laird-Packard 
administration  lielieved  that  large  research  and  development  programs  are  impeded  by  rigid 
fixed-price  contracts  because  they  reduce  the  government’s  ability  to  observe  what  is 
happening  during  the  life  of  the  contract. 

More  recently,  in  the  1980s,  Navy  Secretary  John  Lehman  advocated  fixed-price 
development  as  a  means  of  saving  the  government  money  and  shifting  risk  from  the 
government  to  the  contractor.  The  policy  was  adopted  for  a  number  of  development 
programs,  not  all  of  them  Navy,  and  was  successful  in  shifting  risk  to  the  contractors,  with 
the  result  that  several  contractors  have  sustained  huge  losses  on  fixed-price  development 
contract.  In  reaction.  Congress  precluded  the  use  of  fixed-price  development  contracts  that 
exceed  $10  million  without  approval  from  the  Under  Secretary  of  Defense,  in  the  1989 
National  Defense  Authorization  Act. 


X-6 


2.  Benefits  and  Weaknesses 

In  theory,  the  DoD  selects  a  contract  type  that  will  provide  a  reasonable  distribution 
of  risk  between  government  and  industry.  Fixed-price  contracts  provide  the  greatest  risk  to 
contractors,  and  also  award  the  highest  profit  rate  [59].  Firm-fixed-price  contracts  closely 
resemble  commercial  contracts.  The  contractor  takes  all  the  financial  risk,  and  any  profits 
depend  on  how  well  the  contractor  controls  costs.  In  this  way,  an  FFP  contract  provides 
the  contractor  with  an  incentive  to  avoid  waste. 

Another  great  advantage  of  FFP  contracts  for  the  government  is  that  they  are 
relatively  easy  and  inexpensive  to  administer.  FFP  also  benefits  the  contractor:  the 
government  does  not  monitor  contractor  costs,  so  the  contractor  does  not  have  to  conform 
accounting  methods  to  DoD  audit  procedures.  Administrative  costs  are  therefore  lowered. 

For  the  contractor,  the  expected  benefits  of  the  firm -fixed-price  contract  are  as 
follows: 

•  Higher  profit  potential, 

•  Minimum  government  contfol,  and 

•  Minimum  government  auditing. 

On  the  other  hand,  the  contractor  may  have  to  assume  all  the  financial  and  technical 
risk  and  the  risk  of  greater  liability  for  work  being  performed  [59].  This  is  particularly  a 
problem  in  development,  when  the  design  is  not  yet  established. 

Realistically,  two  important  conditions  should  exist  before  a  firm-fixed-price 
contract  is  negotiated: 

•  Reasonably  definite  design  or  performance  specifications  must  be  available, 
and 

•  The  contracting  parties  must  be  able  to  establish  at  the  outset  prices  that  are 
judged  to  be  fair  and  reasonable. 

In  formally  advertised  procurements,  the  existence  of  definite  specifications  and 
adequate  competition  satisfies  these  conditions.  Even  when  price  competition  is  not 
present,  a  firm-fixed-price  contract  may  be  appropriate  if  one  of  the  following  conditions 
exists: 

•  Historical  price  comparisons  can  be  made, 

•  Available  cost  or  pricing  data  permit  realistic  estimates  of  probable  performance 
and  costs,  and 
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•  Contract  performance  uncertainties  can  be  so  clearly  identiHed  that  their  impact 
on  price  can  be  evaluated. 

When  none  of  these  conditions  exists,  the  use  of  a  fixed-price  contract  with  an 
incentive  feature,  or  a  cost-reimbursement  type  of  contract,  is  normally  considered  more 
appropriate.  Fixed-price  contracts  are  often  used  for  late  production  lots,  when  the  system 
and  production  methods  are  well-defined. 

3.  Analysis 

Because  there  is  relatively  little  production  experience  in  these  firm-fixed-price 
development  programs,  our  conclusions  are  tentative.  Our  analysis  of  10  fixed-price  versus 
91  non-fixed-price  programs  indicates  that  the  development  cost  growth  of  the  firm-fixed- 
price  development  programs  is  47  percentage  points  higher  than  that  of  the  non-fixed  price 
ones.  The  limited  production  cost  growth  data  we  have  shows  a  mix  of  outcomes  for  fixed- 
price  development  programs;  on  average,  their  cost  growth  is  much  higher. 

The  development  cost  and  schedule  outcomes  for  FPD  programs  are  presented  in 
Table  X-3.  As  can  be  seen  from  the  table,  development  outcomes  for  FPD  programs  have 
varied  widely.  Both  the  V-22  and  the  T-AO-187  indicate  negative  development  cost 
growth.  The  T-AO- 1 87’s  low  cost  growth  may  reflect  low  technical  risk,  while  the  V-22  is 
a  relatively  new  program  and  may  not  have  had  time  for  cost  growth  to  accumulate.  The 
E6A,  also  a  new  program,  has  only  three  years  of  production  experience.  Nevertheless, 
most  of  the  FPD  programs  have  high  development  cost  growth.  Particularly  high  are 
Cheyenne  (109  percent)  and  JTIDS  (319  percent). 

The  development  cost  and  schedule  outcomes  of  FPD  programs  are  compared  to 
non-FPD  programs  in  Table  X-4. 

4 .  Findings  and  Recommendations 

Fixed-price  development  contracts  do  not  solve  the  problems  of  cost  growth  and 
schedule  slippage.  Most  fiim-fixed-price  contracts  examined  were  written  for  programs  in 
which  the  element  of  uncertainty  was  high  (AMRAAM  and  V-22,  for  example).  When  a 
contractor  fails  to  perform,  the  government  often  amends  the  contract  and  allows  an 
increase  in  price.  In  some  cases,  a  contractor  who  fails  to  perform  may  be  required  to 
accept  some  loss  along  with  the  contract  change.  For  example,  contract  changes  forced 
contractors  to  absorb  losses  on  the  F-1 1 1,  SRAM,  AMRAAM,  T-45,  C-17,  and  AH-56A 
programs. 
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Table 

X-3.  Outcomas  for  FPD  Programs 

Systems 

Years  of 
Production 

DCG 

TPCG 

DSG 

V-22 

— 

-11% 

— 

0% 

Cheyenne 

— 

109% 

— 

0% 

JTIDS 

— 

319% 

— 

65% 

F-14D 

1 

27% 

— 

0% 

T4.STS 

2 

— 

— 

— 

AMRAAM 

2 

46% 

— 

96% 

E-6A 

3 

12% 

-7% 

31% 

Slinger-RMP 

4 

30% 

— 

63% 

Sgt.  York 

7 

29% 

203% 

15% 

T-AO-IS? 

7 

-3% 

-8% 

5% 

F-14A 

19 

45% 

29% 

16% 

Note;  Cost  growth  figures  are  weighted  by  the  current  estimate. 

Table  X-4. 
With  and 

Outcomas  for  Programs 
Without  FPD  (Parcant) 

Outcome 

With 

Without 

Measure 

FPD 

FPD 

DCG 

mmiM 

I'PCG 

44(78) 

DSG 

35  (91) 

DOG 

36(9) 

7(89) 

Notes;  Numbers  in  parentheses  are  numbers  of 
programs.  Cost  growth  figures  are  weighted  by 
the  current  estimate. 


Based  on  our  interviews  with  the  government,  it  appears  that  FPD  contracts  are 
seldom  executed  as  planned  and  have  to  be  reopened.  The  government  has  difficulty  with 
this,  because  there  is  no  planned  budget  to  address  the  problems.  If  there  are  problems  in 
development,  either  the  development  contract  is  renegotiated,  or  there  is  high  cost  growth 
in  production,  or  both. 

In  short,  firm-fixed-price  contracts  have  not  been  used  effectively  in  development 
programs.  They  have  not  been  successful  in  high-value,  high-cost,  high-risk,  long-term 
programs. 

Since  most  of  the  SAR  programs  are  high  risk,  it  is  not  appropriate  to  use  FPD  in 
these  programs.  FPD  contributes  to  cost  growth  in  development  and  in  the  total  program. 
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XL  CONCLUSIONS  AND  RECOMMENDATIONS 


Acquisition  programs  have  had  varying  degrees  of  success  in  developing  and 
producing  weapon  systems  on  time  and  within  budget.  On  average,  total  program  cost 
growth  was  47  percent — 28  percent  in  development  and  56  percent  in  production. 
Development  schedules  (Milestone  II  to  IOC)  grew  by  an  average  of  about  one-third  over 
the  plan.  There  is,  of  course,  great  variability  in  these  measures  depending  on  program 
characteristics  and  management  strategies. 

The  equipment  types  with  the  highest  total  program  cost  growth  were  tactical 
munitions  (103  percent).  The  lowest  cost  growth  was  exhibited  by  ships  (15  percent) 
followed  by  satellites  (20  percent).  The  highest  development  cost  growth  was  for  vehicles 
(104  percent).  The  lowest  development  cost  growth  measure  was  for  strategic  missiles 
(5  percent). 

Development  schedules  are  particularly  important  for  readiness  considerations — the 
system  should  be  ready  when  planned.  The  groups  with  the  highest  development  schedule 
growth  were  vehicles  and  air-launched  tactical  munitions  while  the  lowest  was  tactical 
aircraft. 

We  would  expect  that  programs  that  succeed  very  similar  systems  (dubbed 
modification  programs,  or  mods)  would  have  lower  cost  growth  than  completely  new 
programs.  Here,  there  were  some  surprising  results.  As  we  expected,  mods  overall  had 
lower  cost  growth  than  new  systems.  For  some  equipment  types,  however,  the  technical 
difficulty  of  modification  programs  is  underestimated  just  as  badly  as  that  of  new 
programs.  Tnis  was  particularly  true  for  air-launched  tactical  munitions  in  development  and 
for  vehicles  and  elecU'onic  aircraft  in  production. 

Acquisition  managers  should  consider  making  more  use  of  prototyping. 
Prototyping  before  FSD  greatly  reduced  development  cost  growth.  The  major  goal  of 
prototyping  is  to  reduce  technical  risk.  However,  we  see  the  benefits  in  reducing  cost 
growth.  This  greater  predictability  of  development  cost  enhances  the  DoD’s  credibility  in 
developing  support  for  further  investments  in  a  program.  This  study  did  not  specifically 
address  the  style  of  prototyping  currently  under  discussion,  the  increased  use  of  advanced 
technology  demonstrators  without  production.  More  study  is  needed  of  this  new  strategy. 
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However,  our  results  do  have  one  clear  implication  for  the  “new  prototyping”  issue.  The 
research  we  have  undertaken  clearly  shows  the  practical  value  of  the  SAR  reporting  system 
in  providing  data  evaluating  acquisition  programs.  While  the  SARs  have  imperfections, 
they  are  extremely  valuable.  The  use  of  advanced  technology  demonstrators  to  keep 
programs  from  proceeding  to  FSD  should  not  diminish  the  importance  of  cost  and  schedule 
estimating,  reporting,  and  monitoring. 

Contract  incentives,  if  successful,  are  an  inexpensive  way  to  induce  contractors  to 
reduce  costs.  Our  results,  which  showed  that  contract  incentives  work  well  in 
development,  were  intuitively  appealing,  but  our  measurement  was  very  simplistic.  We 
believe  that  the  DoD  should  undertake  further  study  of  the  effectiveness  of  the  different 
types  of  contract  incentives  using  more  sophisticated  measurement  techniques  and  the  times 
they  are  applied  in  the  program. 

Multi-year  procurement  appears  to  be  a  successful  initiative,  as  practiced  by  the 
DoD  using  congressional  guidelines.  However,  our  results  do  not  guarantee  success  if 
multi-year  procurement  is  increased.  MYP  implies  some  protection  from  budgetary 
pressures,  and  it  is  impossible  to  offer  such  protection  to  all  programs.  To  a  certain  extent, 
the  programs  that  were  procured  using  MYP  were  allowed  this  privilege  because  they  were 
already  successfully  managed. 

Design-to-cost  needs  to  be  implemented  early  to  be  successful;  otherwise,  it  does 
not  make  sense.  The  record  of  dual-sourcing  is  mixed  but  favorable  for  ships  and  tactical 
munitions,  and  the  techniques  for  evaluating  its  impact  have  not  always  been  carefully 
applied.  The  era  of  fixed-price  development  in  the  early  1980s,  is  now  over.  Our  results 
indicate  that  it  should  not  be  revived.  Neither  should  total  package  procurement. 

DoD  can  use  this  information  to  target  the  classes  of  programs  that  showed  the 
highest  cost  and  schedule  growth — the  tactical  munitions,  ground  combat  programs,  and 
electronics/avionics  programs — for  increased  management  attention. 

When  making  decisions  about  future  programs,  DoD  can  use  the  cost  and  schedule 
records  of  past  programs  of  the  same  equipment  type,  or  programs  in  the  same  time  period, 
or  programs  using  similar  acquisition  strategies.  For  this  reason,  it  is  important  that  the 
database  be  kept  up  to  date  with  each  annual  release  of  the  Selected  Acquisition  Reports. 
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ACQUISITION  PROGRAM  DATABASE 


APPENDIX  A 


VARIABLE  DEFINITIONS  AND 
ACQUISITION  PROGRAM  DATA 


Tabl0  A-1.  Damographics  Variablaa 


Variable 

Definition 

89ID 

Assigned  ID  numbers 

Full  Name 

Program  name 

EQTYPE 

Code  for  Equipment  Type  (Volume  I  classification); 

0 — Satellites 

1 —  ^Tactical  Aircraft 

2 —  ^Electronic  Airiiaft 

3 —  Other  Aircraft 

4 —  Helicopters 

5 —  Air-Launched  Tactical  Munitions 

6 —  Surface  Launched  Tactical  Munitions 

7 —  All  Electronics/Avionics 

8 —  Strategic  Missiles 

9 —  Vehicles 

10 —  Ships 

EQTYPE2 

Ctxle  for  Equipment  Type  (Volume  II  classification): 

0 — Satellites 

1 —  ^Tactical  Aircraft 

2 —  ^ElecU'onic  Aircraft 

3 —  Other  Aircraft 

4 —  Helicopters 

5 —  Air-Launched  Tactical  Munitions 

6 —  Surface  Launched  Tactical  Munitions 

7 —  ^Elecb'onics/Avionics 

8 —  Strategic  Missiles 

9 —  Vehicles  and  Ground  Combat  Programs 

10 —  Ships  and  Ship  Elecu-onics 

BASEYR 

Base  year  used  in  SARs 

FSDST 

FSD  start  year 

SVCE 

Code  for  lead  Service: 

1 —  Army 

2 —  ^Navy 

3 —  Air  Force 

MOD 

Ctxle  to  indicate  Modification  (vs.  New)  programs; 

0 — New  program 

1 — Mod  program 
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TabI*  A-2.  StatSitlea  Summary  Variablaa 


Variable 

Definition 

DCG 

Deveiopoicni  CostGrowtli  Iiuiex 

DSC- 

Development  Schc();ile  Growth 

DQG 

Development  Qttantity  Growth 

PCG 

Production  Cost  Growth  Index 

PSG 

Production  Schedule  Gtowth 

PQG 

Production  Quantity  Gtowth 

TPCG 

Total  I^ram  Cost  Growth  Index 

Table 

A-3.  Cost  and  Quantity  Data  Variables 

Variable 

Definition 

DQ_CE 

Current  Estimate  of  Development  Quantity 

DC_DE 

Development  Estimate  of  Development  Cost,  as  stated 
in  SAR  (base  year) 

DC.VICE 

Current  Estimate  of  Development  Cost  through  end  of 
development  of  first  version,  as  calculated  from  SAR 
(base  year  dollars) 

PQ_CE 

Current  Estimate  of  Production  Quantity 

PC_DE 

Development  Estimate  of  Production  Cost,  as  stated  in 
SAR  (base  year) 

MC.DE 

I>evelopmeni  Estimate  of  Military  Construction  costs 
(base  yecr) 

TPC.DE 

Development  Estimate  of  Total  Program  costs  (base 

year)  a  (DC„DE)  -t-  (PC.DE)  (MC_DE) 


Table  A-4.  Schedule  Data  Variables 


Variable 

Definition 

M2_DE 

Development  Estimate  of  Milestone  II  date 

M2_CE 

Cunent  Estimate  of  Milestone  II  date 

IOC_DE 

Development  Estimate  of  IOC  date 

10C..CE 

Current  Estimate  of  IOC  date 

M3_DE 

Development  Estimate  of  Milestone  III  date 

M3_CE 

Current  Estimate  of  Milestone  III  date 

P_END.DE 

Develc^ment  Estimate  of  Production  End  date 

p_end_ce 

Current  Estimate  of  Production  End  dale 

DS_DE 

Development  Estimate  of  Eievelopment  Schedule 
duration  (in  months)  =  (10C_DE)  -  (M2_DE) 

DS_CE 

Current  Estimate  of  Development  Schedule  duration  (in 
months)  =  (lOC.CE)  -  (M2_CE) 

PS_DE 

Develqiment  Estimate  of  Production  Schedule  duration 
(in  months)  =  (P_END_DE)  -  (M3_DE) 

PS_CE 

Current  Estimate  of  Production  Schedule  duration  (in 
months)  =  (P_END_CE)  -  (M3_CE) 

TS 

Total  Schedule  (in  months)  =  (DS_CE)  +  (PS_CE) 

STRETCH 

Program  Stretch  =  (PSG)/(P(X'i) 
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Tabl«  A-S.  Acquisition  initiatives  Variabias 


Variable 

I>eriniiion 

A  series  of  dummy 
variables  (v/itli  one 

0 — if  initiative  does  not  apply 

exception,  as  noted 
below) 

1 — if  initiative  applies  to  program 

PRO 

Prototyping 

C.FSD 

Competitive  FSD  (Dual  or  Multiple  Source) 

C_PROD 

Competitive  Production  (Dual  Source) 

DTC 

Dcsign-To-Cost 

MYP 

Multi-Year  Procurement 

MYP2 

Multi-Year  Procurement  description 

FPD 

1 —  MYP  in  first  version 

2 —  Immature  MYP  in  first  version,  <  3  years  MYP 
experience 

3_MYP  Candidate 

4— MYP  in  later  version 

0 — All  others 

Fixed-Price  Development 

TPP 

Total  Package  Procurement 

I_FSD 

Contract  Incentives  in  FSD 

LPROD 

Conuact  Incentives  in  Production 

Tabie  A-6.  Damographica  Data 


89ID 

Full  Name 

EOTYPE  EOTYPE2 

BASEYR 

FSDST 

SVCE 

MOD 

1 

Osprey  (V-22) 

3 

3 

86 

86 

2 

0 

2 

T45TS 

3 

3 

84 

84 

2 

0 

3 

B-IA 

3 

3 

70 

70 

3 

0 

4 

C-5B 

3 

3 

80 

82 

3 

■■ 

5 

C-17A 

3 

3 

81 

85 

3 

0 

6 

C-5A 

3 

3 

65 

65 

3 

0 

7 

B-IB 

3 

3 

81 

82 

3 

1 

8 

FB-lllA 

3 

3 

66 

66 

3 

0 

9 

AV-8A 

1 

1 

70 

70 

2 

0 

10 

F-5E 

1 

1 

71 

72 

3 

0 

11 

F-15A/B 

1 

1 

70 

70 

3 

0 

12 

F-16 

1 

1 

75 

75 

3 

0 

13 

F-14D 

1 

1 

89 

84 

2 

1 

14 

F-14A 

1 

1 

69 

69 

2 

0 

15 

AV-8B 

1 

1 

79 

79 

2 

1 

16 

A- 10 

1 

1 

70 

73 

3 

0 

17 

F/A-18 

1 

1 

75 

76 

2 

0 

18 

E-6A 

2 

2 

82 

83 

2 

0 

19 

E-3A 

2 

2 

70 

70 

3 

0 

20 

EF-lllA 

2 

2 

73 

75 

3 

1 
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TabI*  A-6.  Damographics  Data  (Continuod) 


89ID  Full  Name 

EOTYPE  EOTYPE2 

BASEYR 

FSDST 

SVCE 

MOD 

21 

E-2C 

2 

2 

68 

70 

2 

1 

22 

EA.6B 

2 

2 

68 

68 

2 

0 

23 

P-3C 

2 

2 

68 

65 

2 

1 

24 

LAMPS  Mark  III 

2 

2 

76 

77 

2 

0 

25 

E-4 

2 

2 

74 

73 

3 

0 

26 

S-3A 

2 

2 

68 

69 

2 

0 

27 

Chinook  (CH-47D) 

4 

4 

75 

75 

1 

1 

28 

Kiowa  (OH-58D) 

4 

4 

82 

81 

1 

0 

29 

Blackhawk  (UH-60A/L) 

4 

4 

71 

72 

1 

0 

30 

Apache  (AH-64A) 

4 

4 

72 

76 

1 

0 

31 

Cheyenne 

4 

4 

66 

66 

1 

0 

32 

Phoenix  (AIM-54A) 

5 

5 

63 

62 

2 

0 

33 

AMRAAM 

5 

5 

78 

82 

3 

0 

34 

Hellfire 

5 

5 

75 

76 

1 

0 

35 

HARM 

5 

5 

78 

78 

2 

0 

36 

Sparrow  (AIM-7F) 

5 

5 

68 

66 

2 

1 

37 

TOW 

5 

5 

66 

63 

1 

0 

38 

Sidewinder  (AIM-9L) 

5 

5 

71 

71 

2 

1 

39 

TOW2 

5 

5 

84 

78 

1 

1 

40 

Haipocin 

5 

5 

70 

73 

2 

0 

41 

Maverick  (D/G) 

5 

5 

75 

76 

3 

1 

42 

Sparrow  (AIM-7E) 

5 

5 

69 

60 

2 

1 

43 

Sparrow  (AIM-7M) 

5 

5 

78 

78 

2 

1 

44 

Sidewinder  (AIM-9M) 

5 

5 

89 

76 

2 

1 

45 

Phoenix  (AIM-54C) 

5 

5 

77 

77 

2 

1 

46 

ADDS 

7 

7 

83 

85 

1 

0 

47 

MLS 

7 

7 

82 

88 

3 

0 

48 

JTIDS 

7 

7 

81 

81 

3 

0 

49 

JSTARS 

7 

7 

83 

84 

3 

0 

50 

WIS 

7 

7 

82 

85 

3 

0 

51 

SINCGARS 

7 

7 

84 

78 

1 

0 

52 

ASPJ 

7 

7 

84 

81 

2 

0 

53 

LANTIRN 

7 

7 

80 

80 

3 

0 

54 

TRl-TAC 

7 

7 

76 

75 

3 

0 

55 

OTH-B 

7 

7 

82 

82 

3 

0 

56 

DMSP 

0 

0 

75 

76 

3 

0 

57 

Navsuu-  GPS 

0 

0 

79 

79 

3 

0 

58 

DSP 

0 

0 

78 

67 

3 

0 

59 

DSCS  111 

0 

0 

77 

76 

3 

0 

60 

Improved  Hawk 

6 

6 

89 

64 

1 

1 

61 

Shillelagh 

6 

6 

64 

59 

1 

0 

62 

MK-48  ADCAP 

6 

6 

89 

82 

2 

1 

63 

MK-50 

6 

6 

84 

83 

2 

0 

64 

MK48 

6 

6 

72 

68 

2 

0 

65 

Stinger  (B/P) 

6 

6 

72 

75 

1 

0 

66 

Copperliead 

6 

6 

75 

75 

1 

0 

A4 
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Tablo  A-6.  Damographics  Data  (Continuod) 


89ID 

Full  Name 

EOTYPE  EOTYPE2 

BASEYR 

FSDST 

SVCE 

MOD 

67 

5"  Guided  Projectile 

6 

6 

77 

77 

2 

0 

68 

Stinger  RMP 

6 

6 

83 

84 

1 

1 

69 

Dragon 

6 

6 

66 

66 

1 

0 

70 

Pershing  11 

6 

6 

79 

79 

1 

1 

71 

Patriot 

6 

6 

72 

72 

1 

0 

72 

Standard  Missile  2 

6 

6 

84 

72 

2 

1 

73 

Lance 

6 

6 

70 

67 

1 

0 

74 

Peacekeeper 

8 

8 

82 

78 

3 

0 

75 

GLCM 

8 

8 

77 

78 

5 

I 

76 

Tomahawk 

8 

8 

77 

77 

2 

1 

77 

SRAM  11 

8 

8 

83 

87 

3 

0 

78 

Minuteman  II 

8 

8 

69 

65 

3 

1 

79 

Trident  11  Missile 

8 

8 

83 

83 

2 

1 

80 

Small  ICBM 

8 

8 

84 

86 

3 

0 

81 

ALCM 

8 

8 

77 

77 

3 

0 

82 

SRAM 

8 

8 

66 

66 

3 

0 

83 

Minuteman  III 

8 

8 

67 

66 

3 

1 

84 

Condor 

5 

5 

70 

66 

2 

0 

85 

Maverick  (A) 

5 

5 

68 

68 

3 

0 

86 

ASAS/ENSCE 

7 

9 

86 

84 

1 

1 

87 

Bradley  M2/M3 

9 

9 

72 

72 

1 

0 

88 

FAADS_L-R 

6 

9 

89 

86 

1 

0 

89 

Ml  Tank 

9 

9 

72 

76 

1 

0 

90 

MLRS 

6 

9 

78 

77 

1 

0 

91 

M60  A2 

9 

9 

65 

65 

1 

1 

92 

M198  Howitzer 

6 

9 

72 

70 

1 

0 

93 

Roland 

6 

9 

75 

75 

1 

0 

94 

Sergeant  York 

6 

9 

78 

78 

1 

0 

95 

AN/BSY-2 

7 

10 

86 

87 

2 

0 

96 

SURTASS/T-AGOS 

10 

10 

75 

74 

2 

0 

97 

CG47 

10 

10 

78 

78 

2 

1 

98 

DDG51 

10 

10 

87 

83 

2 

0 

99 

FFG  7 

10 

10 

73 

72 

2 

0 

100 

LHA 

10 

10 

69 

69 

2 

0 

101 

LHD  1 

10 

10 

82 

82 

2 

1 

102 

LSD  41 

10 

10 

81 

78 

2 

1 

103 

LSD41  Cargo 

10 

10 

88 

87 

2 

1 

104 

SSN  21 

10 

10 

85 

85 

2 

0 

105 

SSN  688 

I(; 

10 

71 

68 

2 

0 

106 

T-AO  187 

10 

10 

84 

81 

2 

0 

107 

/M3GIS 

7 

10 

70 

69 

2 

0 

108 

AN/BSY-1 

7 

10 

84 

83 

2 

0 

109 

DDG  903 

10 

10 

70 

70 

2 

0 

Table  A-7.  Statistics  Summary  Data 


89ID  Full  Name  DCG  DSG  DOG  PCG  PSG  PQG  TPCG 


1 

Osprey  (V-22) 

0.89 

1.00 

• 

• 

2 

T45TS 

. 

3 

B-IA 

1.10 

1.17 

0.80 

. 

4 

C-5B 

0.77 

1.00 

1.00 

0.76 

5 

C'17A 

1.39 

1.16 

1.00 

6 

C-5A 

2.15 

1.19 

0.66 

1.77 

7 

B-IB 

1.17 

1.00 

1.00 

0.96 

0.73 

1.00 

0.99 

8 

FB-lllA 

2.57 

1.42 

1.00 

1.79 

0.29 

1.83 

9 

AV-8A 

0.99 

0.96 

0.99 

10 

F-5E 

1.05 

1.06 

1.00 

0.79 

1.27 

1.79 

0.88 

11 

F-15A/B 

l.O’’ 

1.03 

1.00 

1.20 

2.25 

1.74 

1.16 

12 

F-16 

1.20 

0.98 

1.00 

1.21 

4.51 

4.61 

1.21 

13 

F-14D 

1.27 

1. 00 

1.00 

14 

F-14A 

1.45 

1.16 

2.00 

1.25 

3.27 

1.26 

1.29 

15 

AV-8B 

1.28 

1.01 

1.00 

0.86 

1.47 

0.82 

0.92 

16 

A-10 

1.27 

1.08 

0.71 

1.34 

1.72 

1.00 

1.33 

17 

F/A-18 

1.15 

1.08 

1.00 

1.43 

1.81 

1.45 

1.38 

18 

E-6A 

1.12 

1.31 

1.00 

0.85 

0.62 

1.07 

0.93 

19 

E-3A 

1.37 

1.16 

3.00 

1.19 

2.56 

0.74 

1.25 

20 

EF-lllA 

2.10 

1.70 

1.00 

1.62 

0.87 

1.00 

1.73 

21 

E-2C 

1.61 

0.76 

1.00 

1.34 

1.37 

3.61 

1.41 

22 

EA-6B 

1.26 

1.00 

1.00 

1.59 

2.76 

1..57 

1.50 

23 

P-3C 

1.22 

1.00 

1.17 

1.79 

3.04 

1.19 

24 

LAMPS  Mark  III 

1.04 

1.00 

1.00 

1.37 

2.04 

1.00 

1.29 

25 

E-4 

1.88 

1..59 

1.00 

0.70 

1.00 

0.50 

1.11 

26 

S-3A 

1.09 

1.00 

0.67 

1.36 

1.00 

0.95 

1.30 

27 

Chinook  (CH-47D) 

1.13 

1.06 

l.CO 

1.34 

1.49 

1.31 

1.32 

28 

Kiowa  (OH-58D) 

0.98 

1.19 

1.00 

1.34 

1.00 

0.42 

1.30 

29 

Blackhawk  (UH-60A/L) 

1.01 

1.05 

0.63 

1.28 

2.29 

2.04 

1.23 

30 

Apache  (AH-64A) 

1.20 

1.49 

1.00 

1.85 

0.94 

1.51 

1.65 

31 

Cheyenne 

2.09 

1.00 

1.00 

32 

Phoenix  (AIM-54A) 

1.54 

1.19 

0.82 

1.36 

0.98 

0.98 

1.39 

33 

AMR A  AM 

1.46 

1.96 

0.66 

34 

Hellllre 

1.09 

1.44 

0.95 

1.71 

1.75 

2.31 

1.44 

35 

HARM 

1.25 

1.60 

1.00 

1.10 

2.08 

1.65 

1.13 

36 

Sparrow  (A1M-7F) 

4.25 

2.82 

3.94 

J.58 

1.22 

1.66 

1.75 

37 

low 

1.20 

1.46 

1.01 

1.78 

2.27 

0.59 

1.70 

38 

Sidewinder  (AIM'9L) 

4.89 

2.45 

4.10 

2.07 

2.76 

1.23 

2.25 

39 

TOW2 

1.00 

1.00 

1.00 

0.95 

1.44 

1.24 

0.95 

40 

Harpoon 

0.93 

1.36 

1.00 

1.94 

2.37 

1.31 

1.60 

41 

Maverick  (D/G) 

1.07 

1.98 

0.94 

1.55 

1.36 

0.63 

1.51 

42 

Sparrow  (AIM-7E) 

0.84 

1.00 

1.00 

1.08 

3.11 

0.34 

1.07 

43 

Sparrow  (AIM-7M) 

0,98 

1..50 

1.00 

1.28 

1.61 

1.41 

1.26 

44 

Sidewinder  (AIM-9M) 

2.04 

1.01 

1.94 

1.01 

2.44 

2.27 

1.10 

45 

Phoenix  (A1M-54C) 

1.67 

1,45 

1..50 

1.98 

1.50 

3.52 

1.92 

46 

ADDS 

1.75 

2.27 

1.00 
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Tabla  A-7.  Statistics  Summary  Data  (Continuad) 


89ID  Full  Name 

DCG 

DSG  DOG 

PCG 

PSG 

PQG 

TPCG 

47 

MLS 

, 

48 

JTIDS 

4.19 

1.65 

3.05 

. 

49 

JSTARS 

1.40 

1.00 

1.06 

, 

, 

50 

WIS 

1.25 

2.33 

1.00 

. 

51 

SINCGARS 

. 

52 

ASPJ 

2.31 

1.91 

1.00 

53 

LANTIRN 

0.96 

1.00 

1.20 

1.00 

0.81 

1.16 

54 

TRl-TAC 

1.02 

0.93 

1.00 

0.87 

1..54 

1.19 

0.88 

55 

OTH-B 

l.Il 

1.63 

1.00 

1.10 

2,50 

1.14 

1.07 

56 

DMSP 

1.02 

1.00 

1.22 

1.29 

1.13 

1.15 

57 

Navstar  GPS 

1.03 

1.62 

1.00 

1.24 

1.71 

1.93 

1.11 

58 

DSP 

1.64 

1.00 

1.00 

1.25 

1.40 

1.10 

59 

DSCS  111 

1.84 

1.62 

1.00 

1.70 

0.87 

1.08 

1.74 

60 

Improved  Hawk 

1.87 

1.25 

1.00 

3.07 

3.16 

1.49 

2.75 

61 

Shillelagh 

1.31 

1.05 

1.38 

1.54 

1.44 

0.89 

1.47 

62 

MK-48  ADCAP 

1.08 

1.34 

1.00 

1.00 

1.53 

1.00 

1.02 

63 

MK-50 

1.30 

1.36 

1.00 

64 

MK48 

1.83 

0.89 

0.57 

1.00 

1..59 

0.68 

1.08 

65 

Stinger  (B/P) 

2.02 

1.64 

0.92 

1.84 

0.82 

0.40 

1.87 

66 

Qrpperliead 

1.28 

1.73 

0.78 

2.23 

1.15 

0.19 

2.12 

67 

5"  Guided  Projectile 

1.16 

1.00 

0.65 

, 

68 

Stinger  RMP 

1.30 

1.63 

1.50 

, 

, 

69 

Dragon 

1.88 

2.14 

1.05 

2.72 

1.13 

0.27 

2..59 

70 

Pershing  11 

1.00 

0.83 

0.82 

2.31 

1.46 

0,71 

1.68 

71 

Patriot 

1.40 

1.12 

0.83 

1.78 

1.15 

0.44 

1.68 

72 

Standard  Missile  2 

1.44 

1.00 

1.00 

0.91 

1.67 

1.36 

0.96 

73 

Lance 

1.08 

1.46 

1.09 

1.20 

1.86 

2.00 

1.12 

74 

Peacekeeper 

0.95 

1.00 

1.00 

1,21 

1.90 

0.78 

1.10 

75 

GLCM 

3.48 

1..30 

0.83 

1.62 

1.30 

0.80 

1.81 

76 

Tomahawk 

1.31 

1.48 

0.91 

1.57 

1.29 

3,35 

1.45 

77 

SRAM  11 

78 

Minuieman  11 

1.00 

1.71 

1.00 

1.24 

1.00 

1.14 

79 

Trident  11  Missile 

0.93 

1.04 

0.93 

80 

Small  ICBM 

0.31 

1.00 

0.14 

81 

ALCM 

1.37 

1.34 

0.69 

1.18 

1.69 

0.51 

1.23 

82 

SRAM 

2.80 

2.03 

1.00 

5. .56 

2.14 

3.63 

83 

Minuieman  III 

0.98 

0.87 

0.73 

1.69 

1.13 

1.39 

84 

('ondor 

1.72 

3.00 

1.19 

6.53 

1.12 

0.02 

5.13 

85 

Maverick  (A) 

1.04 

1.34 

0.91 

0.99 

1  14 

1.18 

1.01 

86 

ASAS/ENSCE 

1.49 

1.49 

1.00 

87 

Bradley  M2/M3 

3.16 

1.23 

1.40 

3. .59 

1.00 

1.93 

3..50 

88 

FAADS_L-R 

1.05 

1.00 

1.00 

89 

Ml  Tank 

1.54 

0.94 

1.00 

1.36 

0.46 

0.75 

1.40 

90 

MLRS 

1.02 

1.06 

0.77 

0.88 

1.72 

1.66 

0.92 

91 

M60  A2 

1.28 

3.17 

1.00 

2.21 

0.90 

0.91 

2.15 

92 

Ml 98  Howitzer 

1.35 

1.30 

1.00 

1.29 

0.91 

0.89 

1.31 
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Tablo  A-7.  Statistics  Summary  Data  (Continuad) 


89ID 

Full  Name 

DCG 

DSCi  DOG 

PCG 

PSG 

PQG 

TPCG 

93 

Roland 

1.52 

2.15 

1.00 

4.83 

0.43 

0.15 

4.19 

94 

Sergeant  York 

1.29 

1.15 

1.00 

3.16 

0.49 

0.10 

3.03 

95 

AN/BSY-2 

. 

. 

. 

96 

SURTASS/T-AGOS 

1.78 

1.38 

1.00 

1.63 

1.54 

1.50 

1.68 

97 

CG47 

1.23 

0.92 

1.00 

0.94 

1.14 

1.69 

0.94 

98 

DDC51 

1.39 

1.20 

1.00 

0.94 

2.11 

4.22 

0.99 

99 

FFG  7 

1.40 

1.15 

l.Ou 

1..59 

1.83 

1.02 

1..59 

100 

LHA 

1.00 

1.56 

1.00 

1.58 

1.80 

0..56 

1.57 

101 

LHD  1 

1.09 

1.06 

1.00 

0.94 

1.49 

2.00 

0.94 

102 

LSD  41 

1.09 

1.04 

1.00 

0.92 

1.00 

0.67 

0.92 

103 

LSD4I  Cargo 

104 

SSN  21 

1.35 

1.04 

1.00 

105 

SSN  688 

0.92 

1.66 

1.00 

0.99 

1.60 

6.20 

0.99 

106 

T-AO  187 

0.97 

1.05 

1.00 

0.92 

1.02 

1.06 

0.92 

107 

AEGIS 

1.28 

0.96 

1.00 

108 

AN/BSY-1 

1.41 

1.00 

, 

109 

DDG  963 

1.06 

1.40 

1.00 

1.23 

2.17 

1.03 

1.23 

Note:  A  period  (.)  in  a  cell  denotes  missing  values  or  incomplete  or  classified  data. 


Table  A-8. 

Cost  and  Quantity  Data 

89ID 

Full  Name 

DQ-CE 

DC_DE 

DC  VICE 

PO  CE 

PC  DE 

MC  DE 

TPC  DE 

1 

Osprey  (V-22) 

0 

2.444 

2,170 

20,493 

136 

23,073 

2 

T45TS 

2 

1.1.50 

506 

300 

2,604 

0 

3,7.55 

3 

B-IA 

4 

2.431 

2,671 

7,423 

0 

9,8.54 

4 

C-5B 

0 

0 

50 

5,724 

122 

5,846 

5 

C-17A 

1 

2.704 

3,756 

210 

16,793 

47 

19,.545 

6 

C-5A 

5 

1.042 

1,026 

76 

2,328 

44 

3,413 

7 

B-IB 

0 

2,539 

2,976 

100 

17,961 

0 

20,.5C0 

8 

FB-1 I i A 

1 

85 

219 

76 

1,696 

0 

1.782 

9 

AV-8A 

0 

110 

504 

0 

504 

10 

F-5E 

5 

101 

106 

147 

191 

0 

292 

11 

F-1.5A/B 

20 

1,658 

1,778 

1266 

4,333 

0 

5,991 

12 

F-16 

8 

579 

693 

2999 

3,798 

0 

4,377 

13 

F-14D 

0 

1,465 

1,865 

437 

13,628 

12 

15,104 

14 

F-14A 

12 

900 

1,308 

583 

4,492 

0 

5,391 

15 

AV-8B 

6 

873 

1,113 

276 

4,862 

6 

5,741 

16 

A-10 

10 

282 

3.59 

727 

1,487 

0 

1,768 

17 

F/A-18 

11 

1.438 

1,652 

11.57 

6,561 

18 

8,017 

18 

f:-6a 

1 

293 

328 

15 

1,292 

0 

1,585 

19 

E-3A 

3 

761 

1,040 

31 

1,390 

0 

2,151 

20 

EF-lllA 

2 

84 

177 

40 

295 

0 

379 

21 

E-2C 

2 

129 

208 

101 

402 

0 

.531 

22 

EA-6B 

5 

2.34 

295 

72 

584 

0 

818 

23 

P-3C 

1 

203 

248 

316 

1,091 

0 

1,294 
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Tabl«  A-8.  Cost  and  Quantity  Data  (Continusd) 


89ID 

Full  Name 

DO_CE 

DC_DE 

DC_V1CE 

PO_CE 

PC_DE 

MC_DE 

TPC_DE 

24 

LAMPS  Mark  III 

5 

527 

550 

204 

1,483 

7 

2,017 

25 

E-4 

1 

159 

299 

3 

255 

29 

443 

26 

S-3A 

4 

565 

618 

183 

1,896 

0 

2,461 

27 

Chinook  (CH-47D) 

3 

76 

86 

472 

806 

0 

883 

28 

Kiowa  (OH-58D) 

5 

214 

210 

243 

1,454 

0 

1,668 

29 

Blackhawk  (UH-60AyL) 

10 

358 

361 

2.257 

1.584 

0 

1,942 

30 

Apache  (AH-64A) 

9 

609 

729 

807 

1.283 

0 

1,892 

31 

Cheyenne 

10 

126 

263 

0 

0 

126 

32 

Phoenix  (AIM-54A) 

37 

94 

144 

2.285 

442 

0 

536 

33 

AMRAAM 

111 

562 

823 

24,320 

4.032 

0 

4,594 

34 

Hellfue 

229 

210 

230 

56.716 

278 

0 

488 

35 

HARM 

99 

227 

284 

22,657 

1,455 

0 

1,682 

36 

Sparrow  (AIM-7F) 

134 

25 

106 

16,145 

380 

0 

405 

37 

TOW 

472 

98 

118 

137,275 

629 

0 

727 

38 

Sidewinder  (AIM-9L) 

123 

13 

66 

11,350 

189 

0 

202 

39 

TOW2 

113 

90 

90 

174,532 

2.163 

0 

2,253 

40 

Harpoon 

52 

272 

254 

3,766 

523 

0 

795 

41 

Maverick  (D/G) 

33 

100 

107 

19,733 

895 

0 

995 

42 

Sparrow  (AIM-7E) 

44 

22 

19 

19,661 

1,247 

0 

1,269 

43 

Sparrow  (A1M-7M) 

44 

55 

53 

15,592 

859 

0 

914 

44 

Sidewinder  (AIM-9M) 

134 

66 

134 

16,937 

701 

0 

767 

45 

Phoenix  (AIM-54C) 

45 

74 

123 

2,483 

297 

2 

372 

46 

ADDS 

3 

175 

306 

120 

1,806 

0 

1,982 

47 

MLS 

6 

35 

28 

316 

48 

0 

82 

48 

JTIDS 

168 

309 

1.296 

0 

NA 

309 

49 

JSTARS 

19 

1,185 

1.656 

118 

3,753 

88 

5,026 

50 

WIS 

1 

545 

681 

34 

642 

2 

1,190 

51 

SINCGARS 

123 

154 

191 

364,802 

4,013 

0 

4,168 

52 

ASPJ 

12 

228 

527 

0 

NA 

228 

53 

LANTIRN 

12 

420 

404 

1,067 

1,682 

0 

2,102 

54 

TRI-TAC 

9 

37 

38 

416 

306 

0 

344 

55 

OTH-B 

1 

327 

364 

8 

711 

107 

1,145 

56 

DMSP 

1 

225 

229 

9 

413 

3 

640 

57 

Navstar  GPS 

12 

926 

956 

54 

623 

8 

1,558 

58 

DSP 

4 

401 

657 

21 

2,123 

26 

2,549 

59 

DSCS  III 

2 

134 

247 

13 

313 

0 

447 

60 

Improved  Hawk 

55 

309 

578 

9,823 

863 

1 

1,173 

61 

Shillelagh 

564 

111 

146 

88,260 

246 

0 

357 

62 

MK-48  ADCAP 

48 

1.212 

1,311 

5,021 

16 

6,250 

63 

MK-50 

108 

1.II8 

1.452 

3,609 

9 

4,736 

64 

MK-48 

60 

150 

276 

2,771 

1,540 

0 

1,690 

65 

Stinger  (B/P) 

205 

76 

154 

9,290 

334 

0 

411 

66 

Copperhead 

320 

105 

135 

24,546 

738 

0 

843 

67 

5"  Guided  Projectile 

141 

95 

110 

0 

0 

95 

68 

Stinger  RMP 

9 

36 

46 

59,059 

2,215 

0 

2,251 

69 

Eiragon 

822 

62 

116 

67,561 

343 

0 

404 
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TabI*  A*8.  Coat  and  Quantity  Data  (Continuad) 


89ID 

Full  Name 

DOXE 

DC_DE 

DC  VICE 

PO_CE 

PC  DE 

MC_DE 

TPC_DE 

70 

Pershing  11 

28 

583 

582 

278 

616 

0 

1,198 

71 

Patriot 

5 

1.106 

1.554 

103 

3.121 

40 

4,267 

72 

Standard  Missile  2 

88 

648 

932 

14,677 

5.923 

. 

6,572 

73 

Lanes 

168 

418 

451 

2.126 

220 

0 

638 

74 

Peacekeeper 

20 

6,018 

5.745 

173 

10.292 

325 

16,635 

75 

GLCM 

5 

75 

261 

560 

928 

51 

1,054 

76 

Tomahawk 

74 

783 

1.C25 

3,630 

1,024 

0 

1,806 

77 

SRAM  II 

0 

861 

791 

1.633 

860 

0 

1.721 

78 

Minuteman  II 

48 

1.437 

1.436 

620 

2,467 

351 

4,255 

79 

Trident  II  Missile 

28 

9.057 

8,393 

871 

14,988 

533 

24,578 

80 

Small  ICBM 

3 

9.777 

3.040 

22,207 

1.727 

33,711 

81 

ALCM 

24 

708 

972 

1.763 

2,312 

121 

3,141 

82 

SRAM 

0 

162 

454 

1.500 

63 

0 

225 

83 

Minuteman  III 

44 

1.835 

1,801 

794 

2,764 

75 

4,674 

84 

Condor 

128 

126 

216 

50 

308 

434 

85 

Maverick  (A) 

186 

116 

121 

20100 

215 

1 

332 

86 

ASAS/ENSCE 

. 

. 

87 

Bradley  M2/M3 

21 

98 

311 

2,300 

227 

0 

326 

88 

FAADS_L-R 

0 

13 

13 

1.207 

1,133 

0 

1.145 

89 

Ml  Tank 

13 

423 

649 

2,488 

1.970 

0 

2,393 

90 

MLRS 

504 

261 

268 

4.813 

1,971 

0 

2,233 

91 

M60  A2 

3 

14 

18 

543 

192 

0 

206 

92 

M198  Howitzer 

10 

31 

42 

584 

80 

0 

111 

93 

Roland 

4 

160 

244 

27 

678 

0 

838 

94 

Sergeant  York 

4 

163 

211 

64 

2,043 

0 

2,207 

95 

AN/BSY-2 

0 

1,566 

1.819 

31 

NA 

0 

96 

SURTASSH'-ACiOS 

1 

59 

106 

18 

147 

0 

206 

97 

CG47 

0 

56 

68 

27 

8,958 

14 

9,027 

98 

DDG51 

0 

892 

1,241 

38 

6,794 

26 

7,712 

99 

FFG  7 

0 

14 

20 

51 

2,606 

0 

2,620 

100 

LHA 
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22 

22 
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1,269 

0 

1,291 

101 

LHD  1 
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40 

44 

6 

2,892 

0 

2,932 

102 

LSD  41 

0 

47 

51 

8 

3,177 

0 

3,224 

103 

LSD41  Cargo 

0 

15 

13 

6 

1,335 

0 

1.351 

104 

SSN  21 

0 

1.725 

2,332 

9 

1,425 

84 

3,233 

105 

SSN  688 

0 

5 

5 

62 

5,127 

17 

5,149 

106 

T-AO  187 

0 

16 

15 

18 

2.592 

0 

2,608 

107 

AEGIS 

1 

394 

504 

65 

NA 

0 

108 

AN/BSY-1 

2.027 

2,849 

31 

NA 

0 

109 

DDG  963 

0 

36 

38 

31 

2,372 

0 

2,408 

Note:  A  period  (.)  in  a  cell  denotes  missing  values  or  incomplete  or  classified  data. 
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Table  A-10.  AcquUition  Initiativaa  Data 


89ID 

Full  Name 

PRO 

C  FSD 

C  PROD 

DTC 

MYP 

MYP2 

FPD 

TPP 

I.FSD 

LPROD 
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Osprey  (V-22) 

1 
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. 
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1 
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• 
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0 

0 

1 
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0 

0 
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0 
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C-5B 
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0 
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C-17A 
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0 

0 
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0 

0 

0 

0 

0 

0 

0 

1 

1 

1 

7 

B-IB 

0 

0 

0 

0 

1 

1 

0 

0 

0 

1 

8 

FB-lllA 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

9 

AV-8A 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

10 

F-5E 

1 

0 

0 

0 

0 

0 

0 

0 

1 

1 

11 

F-15A/B 

0 

0 

0 

0 

0 

3 

0 

0 

1 

1 

12 

F-16 

1 

0 

0 

1 

1 

1 

0 

0 

1 

1 

13 

F-14D 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

14 

F-14A 

0 

0 

0 

0 

0 

0 

1 

0 

1 

0 

15 

AV-8B 

1 

0 

0 

1 

1 

2 

0 

0 

1 

1 

16 

A-10 

1 

0 

0 

1 

0 

0 

0 

0 

1 

1 

17 

F/A-18 

0 

0 

0 

1 

0 

3 

0 

0 

1 

1 

18 

E-6A 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

19 

E-3A 

1 

0 

0 

0 

0 

0 

0 

0 

1 

1 

20 

EF-lllA 

0 

0 

0 

1 

0 

0 

0 

0 

1 

0 

21 

E-2C 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

22 

EA-6B 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

23 

P-3C 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

24 

LAMPS  Mark  111 

1 

0 

0 

1 

0 

0 

0 

0 

1 

1 

25 

E.4 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

26 

S-3A 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

27 

Chinook  (CH-47D) 

0 

0 

0 

1 

1 

1 

0 

0 

0 

1 

28 

Kiowa  (OH-58D) 

0 

0 

0 

1 

0 

0 

0 

0 

1 

0 

29 

Blackhawk  (UH-60A/L) 

1 

0 

0 

1 

1 

1 

0 

0 

1 

1 

30 

Apache  (AH-64A) 

1 

0 

0 

1 

0 

2 

0 

0 

0 

1 

31 

Cheyenne 

1 

0 

0 

0 

0 

0 

1 

0 

0 

0 

32 

Phoenix  (AIM-54A) 

1 

0 

0 

0 

0 

0 

0 

0 

0 

1 

33 

AMRAAM 

1 

0 

1 

1 

0 

0 

1 

0 

0 

0 

34 

HeiUlre 

1 

1 

1 

1 

0 

0 

0 

0 

1 

0 

35 

HARM 

1 

0 

0 

1 

0 

3 

0 

0 

1 

1 

36 

Sparrow  (A1M-7F) 

0 

0 

1 

0 

0 

0 

0 

0 

1 

1 

37 

TOW 

1 

0 

1 

0 

1 

1 

0 

0 

0 

1 

38 

Sidewinder  (A1M-9L) 

0 

0 

1 

1 

0 

0 

0 

0 

1 

1 

39 

TOW2 

0 

0 

0 

0 

1 

2 

0 

0 

1 

1 

40 

Harpoon 

1 

0 

0 

1 

0 

3 

0 

0 

1 

1 

41 

Maverick  (D/G) 

1 

0 

1 

1 

0 

2 

0 

0 

1 

1 

42 

Sparrow  (AIM-7E) 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

43 

Sparrow  (AIM-7M) 

1 

0 

1 

1 

0 

0 

0 

0 

1 

1 

44 

Sidewinder  (AIM-9M) 

0 

0 

1 

1 

0 

0 

0 

0 

0 

0 

45 

Phoenix  (A1M-54C) 

0 

0 

1 

0 

0 

0 

0 

0 

1 

1 

46 

ADDS 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 
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Full  Name 

PRO 

C_FSD 

C.PROD 

DTC 

MYP 

MYP2 

FPD 

TPP 

LFSD 

LPROD 

47 

MLS 

0 

0 

0 

0 

0 

0 

0 

, 

48 

JTIDS 

1 

0 

0 

. 

0 

1 

0 

0 

49 

JSTARS 

0 

1 

0 

0 

0 

0 

0 

0 

1 

1 

50 

WIS 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

51 

SINCGARS 

1 

0 

1 

0 

0 

0 

0 

0 

0 

0 

52 

ASPJ 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

53 

LANTiRN 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

54 

TRI-TAC 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

55 

OTH-B 

1 

0 

0 

0 

0 

0 

0 

0 

0 

1 

56 

DMSP 

0 

0 

0 

0 

1 

1 

0 

0 

1 

1 

57 

Navstar  GPS 

0 

0 

0 

1 

1 

1 

0 

0 

1 

0 

58 

DSP 

0 

0 

0 

0 

1 

1 

0 

0 

1 

1 

59 

DSCS  III 

0 

0 

0 

1 

1 

1 

0 

0 

1 

0 

60 

Unproved  Hawk 

0 

0 

0 

0 

1 

2 

0 

0 

1 

1 

61 

Shillelagh 

0 

0 

1 

0 

1 

1 

0 

0 

1 

1 

62 

MK-48  ADCAP 

0 

0 

1 

0 

0 

0 

0 

0 

1 

1 

63 

MK-50 

1 

0 

1 

1 

0 

0 

0 

0 

1 

0 

64 

MK48 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

65 

Stinger  (B/P) 

0 

0 

1 

1 

0 

0 

0 

0 

1 

1 

66 

Copperhead 

1 

0 

0 

1 

0 

0 

0 

0 

0 

1 

67 

5"  Guided  Projectile 

0 

0 

0 

1 

0 

0 

0 

0 

1 

0 

68 

Stinger  RMP 

0 

0 

0 

1 

1 

1 

1 

0 

1 

1 

69 

Dragon 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

70 

Pershing  11 

0 

0 

0 

1 

0 

0 

0 

0 

1 

0 

71 

Patriot 

0 

0 

0 

1 

1 

1 

0 

0 

1 

1 

72 

Standard  Missile  2 

0 

0 

1 

0 

0 

0 

0 

0 

1 

1 

73 

Lance 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

74 

Peacekeeper 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

75 

GLCM 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

76 

Tomahawk 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

77 

SRAM  II 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

78 

Minuteman  II 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

79 

Trident  11  Missile 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

80 

Small  ICBM 

1 

0 

0 

0 

0 

0 

0 

0 

1 

0 

81 

ALCM 

0 

1 

0 

1 

0 

0 

0 

0 

1 

1 

82 

SRAM 

0 

6 

0 

0 

0 

0 

0 

1 

0 

0 

83 

Minuteman  III 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

84 

Condor 

0 

0 

0 

1 

0 

0 

0 

0 

1 

1 

85 

Maverick  (A) 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

86 

ASAS/ENSCE 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

87 

Bradley  M2/M3 

1 

0 

0 

1 

0 

4 

0 

0 

1 

0 

88 

FAADS_L-R 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

89 

Ml  Tank 

1 

1 

0 

1 

0 

4 

0 

0 

1 

1 

90 

MLRS 

1 

1 

0 

1 

1 

1 

0 

0 

1 

0 

91 

M60  A2 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

92 

M 198  Howitzer 

1 

0 

0 

0 

1 

1 

0 

0 

0 

0 

Tabis  A-10.  Acquisition  Initiativas  Data  (Continuad) 
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Full  Name 

PRO 

C_FSD 

C_PROD 

DTC 

MYP 

MYP2 

FPD 

TPP 

I_FSD 

LPROD 

93 

Roland 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

94 

Sergeant  York 

0 

1 

0 

1 

0 

0 

1 

0 

0 

1 

95 

AN/BSY-2 

1 

1 

0 

1 

0 

0 

0 

0 

1 

1 

96 

SURTASS/T-AGOS 

1 

1 

1 

1 

1 

1 

0 

1 

0 

1 

97 

CG47 

1 

0 

1 

0 

0 

0 

0 

0 

1 

1 

98 

DDG51 

0 

0 

1 

1 

0 

0 

0 

0 

NA 

1 

99 

FFG  7 

1 

1 

1 

1 

0 

0 

0 

0 

1 

1 

100 

LHA 

0 

0 

0 

NA 

0 

0 

0 

1 

1 

1 

101 

LHDl 

0 

0 

1 

1 

0 

0 

0 

0 

1 

1 

102 

LSD  41 

0 

1 

1 

1 

0 

0 

0 

0 

1 

1 

103 

LSD41  Cargo 

0 

0 

1 

1 

1 

1 

0 

0 

0 

1 

104 

SSN  21 

1 

0 

1 

0 

0 

0 

0 

0 

0 

1 

105 

SSN  688 

0 

0 

1 

NA 

1 

1 

0 

0 

NA 

1 

106 

1-AO  187 

0 

0 

1 

0 

0 

0 

1 

0 

1 

1 

107 

AEGIS 

1 

1 

0 

0 

0 

0 

0 

0 

1 

1 

108 

AN/BSY-1 

1 

0 

0 

1 

0 

0 

0 

0 

1 

1 

109 

DDG  963 

1 

0 

0 

0 

0 

0 

0 

1 

1 

1 
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APPENDIX  B 

WEIGHTING  OF  COST  GROWTH 


The  average  total  program  cost  growth  for  the  sample  of  programs  in  the  study  is 
SO.  6  percent.  Calculating  a  simple  mean  such  as  this  places  the  emphasis  on  the  program  as 
an  institution,  without  regard  to  size. 

However,  we  may  also  want  to  consider  the  impact  of  cost  growth  on  spending.  In 
that  case,  it  is  important  to  understand  that  a  small  percentage  cost  growth  in  a  large 
program  such  as  the  F-15  amounts  to  more  in  terms  of  dollars  than  a  large  percentage 
growth  in  a  small  program  such  as  the  AIM-9L.  Cost  growth  generally  is  less  in  larger 
programs,  as  shown  in  Figure  B-1.  (In  the  case  of  DE  size,  the  relationship  is  statistically 
significant  at  the  .05  level.)  This  implies  that  analysis  of  weighted  cost  growth  is  desirable 
for  some  applications. 
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Program  Size  (Millions  of  FY  1991  Dollars) 
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Figuro  B-1.  Total  Program  Cost  Growth  by  Program  Slza 
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In  a  prior  study  [1],  we  weighted  cost  growth  by  the  size  of  the  program,  as 
measured  by  the  actual  cost  of  the  planned  quantity.*  Using  the  same  procedure  for  data  in 
this  study  results  in  a  cost  growth  of  47  percent,  which  is  lower  than  the  unweighted  mean 
but  higher  than  some  alternative  procedures.  Part  of  the  reason  for  the  high  value  is  that 
relatively  high  weights  are  given  to  such  programs  as  Roland,  Condor,  and  Sgt.  York  that 
engaged  in  minimal  production.  This  procedure  has  the  advantage  that  it  reflects  the  actual 
cost  of  the  program  that  the  estimator  was  asked  to  estimate. 

An  alternative  procedure  is  to  weight  by  the  planned  cost  of  the  program.  This 
procedure  results  in  a  mean  of  29.4  percent.  It  reflects  the  total  dollars  invested,  without 
regard  to  the  program  as  an  institution.  It  has  the  desirable  property  that  it  yields  the  sum  of 
the  current  estimates  divided  by  the  sum  of  tlie  development  estimates.^ 

Yet  another  alternative  is  to  weight  by  the  latest  current  estimate,  including 
additional  quantity  not  planned  for  and  additional  versions  of  the  system.  Early  studies  of 
cost  growth  (see  Harman  [60  and  61])  used  this  procedure.  This  weighting  procedure  has 
the  desirable  property  of  giving  additional  credit  to  successful  programs  that  went  through 
several  versions.  However,  the  subsequent  versions  may  not  have  been  subjected  to  the 
acquisition  initiatives  whose  impact  we  are  trying  to  measure,  and,  often,  the  line  between 
programs  is  arbitrary.  For  example,  the  F-15  program  is  still  reporting  under  the  title  F-15, 
even  though  it  is  on  its  fifth  version,  while  the  F-14  program  began  reporting  separately 
with  the  F-14D  version. 

Table  B-1  shows  the  mean  values  of  the  key  outcomes,  unweighted  and  weighted 
using  the  three  alternatives  identified  here. 


A  weighted  average  i.s  computed  in  the  following  manner: 

■  Each  value  Vj  i.s  muluplied  by  its  weight  wj . 

•  Both  die  weights  and  die  products  are  added. 

•  llie  sum  of  the  products  is  divided  by  the  sum  of  the  weights  to  obtain  the  weighted  average: 

weighted  average  = - . 

i(”,) 

Weighted  averages  are  affected  by  the  proportions  among  the  weights  but  not  by  the  ab.solute  sizes  of 
the  weights. 

la, 

X  DE  "IDE 

I  I 
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c  Actual  spending  weights  are  approximate  for  development  and  for  total  program, 
since  we  have  actual  development  spending  in  the  database  for  only  26  programs. 
For  the  other  programs,  we  used  cur  current  estimate  weights.  The  production 
weights,  which  represent  the  greatest  part  of  the  total,  are  exact. 


Figure  B-2  shows  the  relative  weights  for  development  cost  growth.  For  a  given 
level  of  cost  growth,  a  horizontal  slice  through  the  chart  indicates  the  relative  weights.  For 
example,  near  the  top  of  the  chart,  in  the  region  of  high  cost  growth,  the  CE  weights  are 
generally  the  furthest  to  the  right,  indicating  the  highest  weights.  In  the  lower  part  of  the 
chart,  the  DE  weights  are  more  likely  to  be  higher. 


Percent  of  Total  Weight 

#  DE  Weights  D  CE  (First  Version)  Weights  A  Total  Spending  Weights 

Figure  B-2.  Development  Cost  Growth  With  Different  Weights 

Figure  B-.T  shows  relative  weights  for  total  program  cost  growth.  Again,  the  same 
relationship  holds. 
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Percent  of  Total  Weight 
•  DE  Weights  □  CE  at  DEO  Weights 

Figure  B-3.  Total  Program  Cost  Growth  With  Different  Weights 

In  this  study,  we  have  emphasized  the  relationships  between  cost  growth  and 
factors  such  as  equipment  type,  program  type,  and  the  management  initiatives.  For  that 
purpose,  it  is  not  so  much  the  average  values  of  cost  growth  as  their  relative  magnitudes 
that  are  important.  We  have  used  the  CE  at  DEQ  weights  as  our  default  weighting  system. 
In  Section  IV,  we  include  the  CE  weights  in  the  major  tables  for  illustrative  purposes.  In 
sections  on  the  initiatives,  we  note  analyses  in  which  weighting  makes  a  difference  in  the 
outcome. 
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production  cost  growth 
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